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ABSTRACT 

This  study  examined  the  productivity  of  the  Automation  of  Procurement  and 
Accounting  Data  Entry  (APADE)  system,  in  a  before/after  quasi-experimental 
design  that  measured  outputs  (workload,  productivity),  inputs  (staff  size,  staff 
grade  structure,  usuage  of  overtime),  and  by-product  social  effects  (annual  leave, 
sick  leave,  and  leave  without  pay)  using  archival  data.  While  workload  decreased  , 
the  procurement  action  lead  time  (PALT)  decreased  by  55%  after  APADE 
implementation.  This  result  was  obtained  as  the  size  of  the  staff  decreased  and 
overtime  usuage  declined  sharply.  The  implementation  of  APADE  streamlined  the 
document  process  significantly  at  the  test  site. 
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I .    INTRODUCTION 

Computer  industry  literature  claims  that  no  longer  is 
automation  synonymous  with  automatic  productivity 
improvement.   Organizations  are  now  struggling  with 
developing  methods  to  measure  productivity  improvement  in 
the  white  collar  sector.   The  Federal  Government  has 
invested  millions  of  dollars  in  the  development  of  automated 
procurement  systems.   However,  little  has  been  done  with 
regards  to  the  measurement  of  change  in  productivity 
resulting  from  the  implementation  of  these  systems. 

Our  intent  in  this  study  is  to  further  develop  a 
methodology  which  can  be  used  to  measure  productivity  for 
automated  procurement  systems.   An  installed  Automation  of 
Procurement  and  Accounting  Data  Entry  (APADE)  system  is 
examined  to  assess  the  impacts  of  office  automation  on 
productivity.   This  system  has  been  in  operation  for  over 
two  years  at  a  Navy  supply  center,  which  we  call  NSC  Duarte . 

APADE  was  designed  to  provide  the  Navy  field  contracting 
system  with  a  standard  automated  procurement  mechanism.   It 
has  been  synonymous  with  automated  procurement  in  the 
Department  of  the  Navy  since  the  1970' s.   Prior  to  the 
current  installation  of  APADE,  NSC  Duarte  was  using  a  system 
called  APADE  II.   APADE  II  was  essentially  a  manager's 
information  system  which  provided  management  statistics  to 


supervisors  upon  request.  For  the  purpose  of  this  study, 
and  for  the  duration  of  this  thesis,  the  term  "pre  APADE" 
will  be  used  when  referring  to  APADE  II. 

The  current  system  implementation  is  called  APADE  and  it 
includes  many  of  the  functions  contained  in  pre  APADE. 
APADE  was  designed  as  a  decision  support  system  for  Navy 
buyers.   Functions  such  as  providing  a  price  history  and 
source  file  allow  the  buyer  access  to  on-line  information 
that  had  previously  only  been  available  on  a  manual  basis. 
The  system  also  provides  an  automatic  document  preparation 
capability,  removing  the  need  to  manually  generate  each 
document.   In  addition,  APADE  provides  automated  document 
control  and  tracking  status  of  procurement  actions.   APADE 
provides  real-time  contracting  information  to  management 
while  allowing  selective  access  to  appropriate  customers. 
Although  APADE  can  now  process  large  contracts,  this  thesis 
will  concentrate  on  contracts  under  $25,000,  procured  by  the 
Small  Purchasing  Branch  (SPB)  at  Duarte   on  the  APADE 
system.   A  description  of  the  procurement  processes  under 
both  APADE  and  pre  APADE  is  provided  in  Appendix  A. 

Our  goal  is  to  measure  specific  organizational  factors 
to  determine  what  effect  APADE  had  on  the  effectiveness  of 
Duarte' s  Small  Purchasing  Branch.   In  doing  this,  we  used 
the  ratio  of  outputs  divided  by  inputs  to  calculate 
productivity.   Productivity  is  enhanced  as  the  proportion  of 
outputs  to  inputs  increases. 


In  this  study,  the  inputs  used  include  staff  size,  grade 
structure,  and  overtime  hours  worked.  (See  Appendix  B)   The 
outputs  are  the  number  of  purchase  orders  processed  and 
Procurement  Action  Lead  Time  (PALT)  measured  in  days.   This 
study  also  examines  social  factors  which  affect 
productivity:  annual  leave  taken,  sick  leave  used,  and  leave 
without  pay  taken.  (See  Appendix  C)   Lastly,  we  evaluate 
productivity  as  a  measure  of  purchase  orders  processed  per 
procurement  labor  hour.  (See  Appendix  D) 1 


Appendixes  B,  C,  and  D  contain  summary  data  per  pay  period. 
For  the  comprehensive  sample  of  raw  data  used  by  this  study  contact 
Professor  William  J.  Haga  at  the  Naval  Postgraduate  School, 
Monterey,  California. 


II.    LITERATURE  REVIEW 

Throughout  the  1980' s  industry  has  attempted  to  automate 
the  white  collar  environment.  It  had  been  assumed  that 
productivity  improvement  would  come  with  automation. 
However,  this  may  not  have  been  the  case  and  many 
corporations  are  now  questioning  office  automation  projects 
more  thoroughly  than  before. 

A.  WHAT  IS  NOT  HERE 

This  study  does  not  attempt  to  consider  the  measurement 
of  productivity  of  knowledge  workers  (professional, 
technical,  administrative,  and  managerial) .   The  focus  of 
our  study  concerns  the  measurement  of  productivity  of 
clerical  workers. 

B.  BASIC  INPUT/OUTPUT  MEASUREMENT 

There  are  numerous  definitions  that  have  been  applied  to 
productivity.   Some  are  vague,  "Productivity  is  a  particular 
type  of  behavior  within  an  organization.   It  may  be  high,  a 
positive  and  desirable  behavior;  or  low,  a  negative  and 
undesirable  behavior."  (Edosomwan,  1987,  p.  5)   Others  are 
relatively  specific  "total  productivity  is  a  ratio  of  all 
measurable  outputs  to  the  sum  of  all  measurable  inputs." 
(Edosomwan,  1987,  p.  5)   However,  the  basic  form  that  most 


definitions  follow  is  that  productivity  is  the  ratio  of  some 
output  to  some  input . 

Bain  (1982)  indicates  that  it  is  not  just  a  simple  ratio 
nor  is  it  a  measure  of  production  or  output  produced.   "It 
is  a  measure  of  how  well  resources  are  combined  and  utilized 
to  accomplish  specific,  desirable  results."   (Bain,  1982, 
p. 3) 

C.   IMMEASURABLE  PRODUCTIVITY 

Measuring  productivity  in  a  white  collar  environment  has 
long  been  considered  difficult  at  best.   Rowe  states  that 
"the  great  nemesis  of  measuring  white  collar  output  has  been 
the  inability  to  quantify  the  end  results  of  the  white 
collar  employee."   (Rowe,  1981,  p.  43)   Even  the  U.  S. 
Department  of  Labor  states  that  white  collar  productivity  is 
so  difficult  to  measure  that  they  only  put  out  that 
statistic  by  industries   (Leeke,  1988) . 

Much  of  the  productivity  to  be  gained  from  white  collar 
workers  is  of  an  intangible/ immeasurable  variety.   Leeke 

(1988)  states  it  is  frequently  a  subjective  measurement, 
often  not  based  on  quantifiable  characteristics.   One  can 
easily  see  the  difficulty  in  measuring  such  unquantif iable 
factors  as  improved  customer  service,  improved  employee 
morale,  and  timely  information  needed  for  decision  making 

(Barclift  and  Linson,  1988) .   Schwartz  (1987)  states  the  if 
there  is  no  reduction  in  headcount  there  are  no  easy  means 


available  for  measuring  white  collar  productivity.   In  fact, 
the  "payoffs  become  more  difficult  to  assess  as  information 
technologies  -  office  systems,  personal  computers,  expert 
systems,  and  application  enhancement  projects  -  are 
increasingly  directed  at  improving  the  performance  of  white 
collar  workers."   (Schwartz,  1987,  p.  47) 

D.  SUBSTITUTING  ATTITUDE  SURVEYS  FOR  INPUT/OUTPUT 

Another  form  of  measurement  that  has  been  used  to 
measure  productivity  as  well  as  system  performance,  usage, 
and  effectiveness  is  the  user  attitude  survey.   Miller  and 
Doyle   (1987)  indicate  that  there  have  been  a  number  of 
instances  where  the  measurement  of  user  satisfaction  has 
been  used  as  a  surrogate  for  overall  effectiveness  of 
information  systems  within  organizations.lt  has  been 
generally  accepted  within  the  academic  MIS  community  that 
user  satisfaction  can  be  correlated  to  information  usage  and 
system  success   Bailey  and  Pearson,  1983) . 

E.  VARIETIES  OF  INPUT/OUTPUT  ANALYSIS 

There  are  a  number  of  ways  in  which  productivity 
improvements  may  be  achieved.   Bain  (1982)  provides  a 
summary: 

1.  Output  increases  while  input  decreases. 

2.  Output  remains  the  same  while  input  decreases. 

3 .  Output  increases  while  input  remains  the  same . 
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To  this  list,  Sink  (1985)  adds  two  other  ways  in  which 
productivity  improvement  may  be  obtained: 


1.  Output  increases  while  input  increases  at  a  slower 
rate . 

2 .  Output  decreases  while  input  decreases  at  a  greater 
rate. 


In  measuring  productivity,  Bain  (1982)  states  that  one 
must  consider  the  interplay  between  various  factors  within 
an  organization.   "While  the  output  may  be  related  to  many 
different  inputs  in  the  form  of  productivity  ratios,  each  of 
the  separate  productivity  ratios  is  influenced  by  a 
combination  of  many  relevant  factors."  (Bain,  1982,  p.  3) 
The  influencing  factors  to  be  considered  range  from  the 
quality  and  availability  of  materials  to  the  attitude  and 
skill  level  of  the  work  force. 

In  his  description  of  productivity  measures,  Sink 
defines  two  basic  categories  of  productivity  measures.   The 
first,  static  productivity  ratios,  involve  output  divided  by 
input  for  a  given  period  of  time.   The  second  category, 
dynamic  productivity  ratios,  are  essentially  particular 
static  productivity  ratios  at  one  point  in  time  divided  by 
the  same  ratio  at  some  prior  period.   This  provides  us  with 
an  index  which  reflects  the  change  in  productivity  between 
different  periods  in  time.   Within  each  category  there  are 
three  different  types  of  productivity  measures.   The  only 


differentiating  characteristic  between  them  is  the  degree  of 
input  that  is  used  in  the  denominator  of  the  equation. 
Partial-factor  measures  include  only  one  class  of  input, 
such  as  labor,  capital,  or  energy.   Multi-factor  measures 
include  more  than  one  class,  while  total-factor  measures 
include  all  classes  of  input.  (Sink,  1985) 

F.   API:  ADMINISTRATIVE  PRODUCTIVITY  INDEX 

Bolte  (1983)  and  his  colleagues  at  Intel  wished  to 
develop  a  quantitative  system  which  would  focus  on  improving 
administrative  productivity  as  well  as  on  reducing  head 
count.  In  searching  the  literature  they  found  that  there  was 
very  little  practical  information  available  on 
quantitatively  controlling  headcount  growth  and  measuring 
and  improving  administrative  productivity.   As  a  result, 
they  set  out  to  develop  a  quantitative  system  for  measuring 
productivity  in  administrative  functions. 

To  begin  with,  Bolte  used  the  definition  of  productivity 
currently  in  use  at  Intel:  output  divided  by  input. 
Specifically,  the  physical  units  of  output  are  divided  by 
the  total  number  of  hours  it  takes  to  produce  them.   It  was 
decided  not  to  use  financial  measures  of  input  or  output, 
such  as  sales  or  revenues,  because  their  "definition  is 
understandable  and  controllable  at  the  line  management 
level,  which  is  where  productivity  improvements  must  take 
place."  (Bolte,  1983,  p.  47) 


Next,  a  set  of  indicators  was  developed  which  were 
affected  by  their  measurement  systems .   The  first  was  to 
establish  a  set  of  quantity  and  quality  indicators.   Each 
department  would  establish  a  goal  for  the  quality  and 
quantity  of  its  work.  Second  the  quality  and  quantity 
indicators  of  one  function  or  site  would  be  compared  with 
those  of  another.   Lastly,  the  ratio  between  supervisors  and 
employees  is  employed  as  a  measure  of  the  ratio  between 
direct  labor  to  indirect  labor  (supervisors) . 

In  an  effort  to  develop  a  quantitative  system  for 
measuring  productivity,  Bolte  conceptualized  administrative 
areas  as  paper  processing  factories  "with  specific  inputs 
and  required  outputs,  much  like  an  assembly  line,  so  that 
production  techniques  can  be  applied."   (Bolte,  1983,  p.  48) 
With  this  in  mind  he  developed  an  Administrative 
Productivity  Index  (API)  which  would  provide  a  means  of 
comparing  productivity  against  an  established  base  line  at 
different  points  in  time.   API  is  calculated  by  dividing 
output  by  labor  hours,  the  result  is  expressed  in  hours  per 
unit  (HPU) .   The  output  measures  must  be  both  physical  and 
countable,  while  input  is  all  of  the  hours  of  work  paid  for 
by  the  organization,  less  vacation,  absenteeism,  and  sick 
leave,  during  the  period  which  the  output  was  generated. 

Once  baseline  API  values  have  been  established 
productivity  may  be  measured  over  time  by  comparing  the  API 
figures  collected  over  a  period.   Productivity  enhancements 
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are  then  obtained  through  simplifying  tasks,  applying 
workload  management  techniques,  and  monitoring  the  API.   The 
labor  requirement  of  a  unit  to  accomplish  its  mission  is 
reduced,  which  reduces  headcount  and  thus  improves 
productivity . 

G.   CONSENSUS  MODEL:   INFERRED  OUTPUT 

A  consensus  model  seeks  to  obtain  agreement  among 
managers  on  the  projected  inferred  benefits  to  be  obtained 
through  the  implementation  of  a  specific  computer  system. 
Schwartz  (1987)  states  that  the  inferred  output  technique  is 
an  alternative  that  may  be  used  to  measure  productivity  when 
a  direct  output  model  cannot  be  developed.   Managers  first 
determine  estimates  of  the  value  of  various  tasks.   This 
information  coupled  with  their  reasoning  is  then  pooled.   A 
consensus  is  then  formed  after  repeated  estimates  and 
sharing  of  information.   The  underlying  assumption  of  this 
model  is  that  an  increase  in  output  will  generate  an 
increase  in  profit.   This  increased  profit  can  then  be  used 
to  infer  an  increase  in  productivity.   General  Telephone  and 
Electronics  (GTE)  has  used  a  consensus  model  in  its 
operations.   The  "consensus  models  are  most  appropriate  for 
judging  potential  payoff  when  there  is  little  quantitative 
basis  for  making  estimates  of  value."   (Schwartz,  1987,  p. 
48) 
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H.   INFERRED  INPUT  MODEL 

Inferred  input  models  "use  projected  increases  in 
efficiency  and  effectiveness  among  workers  rather  than 
actual,  verified  cuts  in  labor  or  head  count."   (Schwartz, 
1987,  p.  48)   Generally,  the  projections  generated  are  based 
on  a  task/time  matrix  that  indicates  the  time  that  people 
spend  on  tasks  and  the  time  savings  offered  by  the 
implementation  of  the  computer  system.   The  Times- 
Savings/Times  Salary  (TSTS)  model  being  used  by  IBM  is  based 
on  research  done  by  Booze,  Allen,  and  Hamilton,  Inc.  People 
first  determine  the  amount  of  time  they  expend  on  activities 
such  as  reading  and  typing.   Any  savings  that  are  gained 
through  the  implementation  of  a  specific  computer  technology 
are  then  multiplied  by  each  employee's  salary. 

The  TSTS  model  is  simple  and  easy  to  use,  making  it  a 
popular  model.   However,  there  are  drawbacks.   Poppel  (1982) 
contends  that  the  TSTS  model  counts  time  saved  on  lower 
value  activities,  such  as  the  work  of  a  clerk,  equivalently 
with  higher  value  activities.   In  other  words,  TSTS  cannot 
distinguish  between  making  a  white-collar  worker  a  better 
manager  or  analyst  and  making  that  worker  a  better  clerk  or 
receptionist.  (Schwartz,  1987) 

I.   WORK  VALUE  ANALYSIS 

Schwartz  (1987)  and  his  colleagues  have  developed  a 
hybrid  model  which  they  named  the  Work  Value  Analysis  (WVA) 
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model.  This  model  not  only  evaluates  the  efficiency  of  white 
collar  workers  but  also  provides  input  on  their 
effectiveness.  Efficiency  is  the  increased  work  accomplished 
in  a  period  of  time,  while  effectiveness  refers  to  doing  the 
right  things.  This  incorporates  the  thought  that  the  members 
of  an  organization  do  not  spend  all  of  their  time  at  work 
performing  primary  activities.   A  large  portion  of  their 
time  includes  performing  supporting  activities,  such  as 
clerical  duties. 

The  WVA  model  uses  two  forms  of  productivity 
improvement : 


1 .  Technology  can  shorten  the  amount  of  time  required  to 
complete  a  given  task  or  it  can  allow  more  of  the  task 
to  be  completed  in  the  same  amount  of  time. 

2.  Technology  can  be  the  basis  for  a  shift  in  a  work 
pattern  that  allows  more  time  to  be  spent  on  primary 
activities  and  less  on  lower  valued  activities  such  as 
support,  clerical,  lost  time.   (Schwartz,  1987,  p. 52) 


The  first  form  identifies  improvements  in  efficiency, 
while  the  second  form  accounts  for  improvements  in  the 
effectiveness  of  an  organization.   By  identifying  its 
primary  or  high  value  activities,  an  organization  can  target 
those  activities  for  improvement  with  the  greatest 
return/payoff . 

Schwartz  contends  that  its  strength  is  in  its  "objective 
determination  of  payoff  when  external  dollar  criteria 
relating  to  profit  of  value  of  work,  other  than  salary, 
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cannot  be  measured  or  otherwise  inferred."   (Schwartz,  1987, 
p.  52) .  However,  WVA  requires  a  significant  investment  in 
time  and  effort  to  implement  and  the  mathematical 
computations  can  become  complex. 

J.   COST  DISPLACEMENT  MODEL 

Schwartz  (1987)  notes  that  direct  input  models,  such  as 
the  cost  displacement  model,  are  useful  when  inputs  can  be 
determined  and  outputs  cannot  be  measured.   In  order  to 
account  for  output,  the  cost  displacement  model  assumes  that 
output  either  remains  the  same  or  increases.   Therefore,  if 
output  remains  the  same  and  inputs  decrease,  such  as  number 
of  workers,  there  is  an  improvement  in  productivity. 

The  largest  plus  for  cost  displacement  models  is  that 
they  are  simple  to  use.   The  only  requirement  is  that  real 
labor  cuts  or  equipment  savings  be  achieved  due  to  the 
introduction  of  new  information  systems.   However,  without  a 
cut  in  labor  or  equipment  savings  cost,  displacement  models 
are  inappropriate. 

K.   NPMM:  NORMATIVE  PRODUCTIVITY  MEASUREMENT  METHODOLOGY 

The  normative  productivity  measurement  methodology 
(NPMM)  seeks  to  obtain  various  output/input  ratios  through  a 

consensus  building  method  called  the  nominal  group  technique 
(NGM) .   Sink  describes  NPMM  as  an  "action-research, 

involvement-participative,  organizational,  development- 
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oriented  approach  to  measuring  productivity."   (Sink,  1985, 
p.  139) 

The  process  is  encapsulated  in  five  stages.   The  first 
stage  uses  the  normative  group  technique  to  generate  a 
prioritized  list  of  the  measures,  ratios,  or  indices  of 
productivity  for  each  of  the  corresponding  organizational 
units.   The  second  stage  involves  converting  those  measures 
into  a  workable  productivity  measurement  system.   This  task 
is  usually  referred  to  productivity  analysts  for 
development.   The  third  stage  includes  briefings,  reviews, 
discussions,  and  revisions  until  a  workable  system  is 
drafted.  The  fourth  stage  integrates  this  new  system  with 
other  performance  measurement  and  control  systems  already 
implemented  within  the  organization.   Stage  five  involves 
the  continuous  monitoring  and  feedback  of  information  based 
on  the  previously  determined  ratios.   (Sink,  1985) 

NPMM  allows  the  participants  to  formulate  the  measures 
to  be  used  in  the  measurement  system.   The  inclusion  of 
organizational  members  provides  the  advantage  of 
establishing  commitment  among  them  which  is  important  is  any 
project  is  to  succeed. 

L.   MFPMM:  MULTIFACTOR  PRODUCTIVITY  MEASUREMENT  MODEL 

Multifactor  productivity  measurement  (MFPMM)  is  a 
consultative,  data  base/accounting  system  oriented 
methodology  which  collects  its  data  from  system 
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documentation  rather  than  people.   It  is  diagnostic  in  a 
passive,  absolute,  and  objective  sense  as  opposed  to  an 
active,  relative,  and  subjective  sense  (Sink,  1985) .   In 
using  the  various  accounting  data  available  within  an 
organization  a  series  of  base  indices  is  generated  which  may 
be  compared  with  the  values  in  the  succeeding  periods 
determine  relative  productivity  improvements/declines. 

The  multifactor  productivity  measurement  model  can  be 
used  to  provide  information  on  a  variety  organizational 
areas.  These  include: 


1.  Obtain  an  overall,  integrated  measure  of  productivity 
for  the  firm. 

2.  Assess  and  evaluate  bottom-line  results  of  specific 
productivity  improvement  efforts. 

3.  Monitor  the  historical  productivity  performance  and 
measure  how  much,  in  dollars,  profits  were  affected  by 
productivity  growth  or  decline. 

4.  Assist  with  setting  productivity  objectives  and  general 
strategic  planning  in  areas  such  marketing  and  cost 
management.   (Sink,  1985) 


MFPMM  and  other  measurement  systems  similar  to  it  are 
currently  being  used  in  industry.   For  instance, 
Westinghouse  has  used  a  similar  methodology  when  measuring 
white  collar  productivity.   Each  department  would  establish 
its  own  measurement  criteria  or  performance  ratios .   This 
information  would  then  be  used  to  calculate  a  composite 
index,  which  is  calculated  by  assigning  weights  to  the 
measures  and  then  combining  the  values  into  a  composite 
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value  (Rowe,  1981) .   These  base  indexed  measures  can  then  be 
used  to  compare  the  relative  change  in  productivity  in 
succeeding  periods. 

M.   PRODUCTIVITY  MAP 

The  productivity  map  by  Pacesetter  Software  has  been 
developed  to  assist  managers  in  establishing  and  measuring 
productivity  goals  of  white  collar  workers.   It  consists  of 
a  multi-step  process  which  establishes  and  tracks  four 
primary  measures  of  productivity:  quantity,  quality, 
timeliness,  and  cost.   (Miller,  1987) 

Step  one  involves  answering  questions  on  the 
characteristics  and  goals  of  the  next  level  up  in  the 
organization.   "Productivity  map  then  prepares  an  overview 
report  that  summarizes  your  department's  mission,  strategic 
direction,  and  critical  success  factors  based  on  these 
answers."   (Miller,  1987,  p.  43)  Questions  are  then  answered 
regarding  your  department's  role  within  the  organization  as 
well  as  your  perceptions  of  what  the  customers'  needs  are. 
These  are  then  compared  with  the  customers'  needs  and 
relative  priorities.   The  final  step  involves  the 
development  of  productivity  measures  which  are  then  used 
with  the  assistance  of  the  program's  database,  graphics, and 
report  generation  abilities  to  track  the  productivity  of  the 
unit . 
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N.   TOTP  (TASK-ORIENTED  TOTAL  PRODUCTIVITY  MEASUREMENT) 

A  total  productivity  measure  uses  the  total  output  with 
respect  to  the  total  input.   The  Task-Oriented  Total 
Productivity  Measurement  Model  (TOTP)  considers  all  possible 
measurable  input  and  output  components.   The  measures  used 
in  this  model  are  developed  into  a  matrix  which  can  be  used 
to  measure  changes  in  productivity. 

There  are  several  steps  involved  in  the  process.   A 
formal  study  of  the  work  environment,  processes,  and 
procedures  is  first  conducted.   The  results  are  then  shared 
with  those  individuals  that  will  be  affected  by  the 
implementation.   The  next  step  involves  the  selection  of  the 
measurement  criteria  to  be  used.   Overhead  expenses  are  then 
allocated  among  the  units  on  either  a  "proportional  basis" 
or  through  a  "complexity  factor."  A  base  period  is  selected 
which  acts  as  a  reference  period  to  compare  the  measurement 
criteria.   Next,  various  forms  and  instruments  are  developed 
to  capture  the  input  and  output  elements  to  be  measured. 
Personnel  are  then  trained  in  the  measuring  processes  and 
data  collection  begins.  Trend  analysis  and  interpretation  of 
the  findings  can  then  begin.   (Edosomwan,  1987) 

There  are  several  advantages  to  be  obtained  by  using  the 
TOTP  model: 


1.  The  indices  can  be  used  in  the  productivity  planning 
and  improvement  phases . 
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2 .  The  productivity  indices  derived  offer  flexibility  in 
that  they  may  change  as  the  task  parameters,  resources 
used,  etc.  change. 

3.  The  indices  are  comparable  over  time  and  can  be  used  to 
measure  the  productivity  of  various  organizational 
elements,  such  as  task,  work  groups,  and  departments. 

(Edosomwan,  1987) 


However,  Bain  (1982)  indicates  that  while  total  factor 
productivity  makes  conceptual  sense,  "it  is  extremely 
difficult  to  identify  and  capture  all  related  inputs  for  any 
component  output  of  the  organization."   (Bain,  1982,  p.  56) 

O.   OBJECTIVES  MATRIX 

Many  organizations  have  found  that  productivity 
measurement  can  be  quite  difficult  and  complex.   In  fact, 
far  too  few  organizations  actively  pursue  effective 
productivity  measurement.   The  Oregon  Productivity  Center 
(OPC)  has  developed  what  it  believes  is  an  easy-to-implement 
methodology  based  on  its  Objectives  Matrix.   The  Objectives 
Matrix  combines  all  of  an  operation's  important  productivity 
criteria  into  one  interrelated  format,  thus  overcoming  the 
problem  of  complexity. 

The  first  step  to  identify  the  key  performance 
indicators.   Once  this  is  done  each  indicator  is  to  be 
thoroughly  defined  and  sources  of  this  information 
identified.   Performance  is  then  assessed  over  time  and  the 
information  obtained  is  used  in  determining  productivity 
objectives.   Next,  each  indicator  is  assigned  an  importance 
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weighting,  usually  100  total  points  distributed  across  the 
indicators  (Felix  and  Riggs,  1983)  .   Improvement /decline  in 
productivity  over  time  is  then  measured  against  the  initial 
index. 

Felix  and  Riggs  (1983)  indicate  that  there  are  several 
advantages  obtained  by  using  the  Objectives  Matrix: 


1.  Measures  are  normalized  by  establishing  a  uniform 
quantitative  rating  system. 

2.  Many  dimensions  of  performance  may  be  followed  at  the 
same  time  providing  a  global  perspective  of  the 
organization . 

3.  The  Objectives  Matrix  focuses  on  results  rather  than  on 
activities. 


P.   SUMMARY 

A  review  of  the  available  literature  revealed  the 
following: 


1 .  There  is  little  documentation  available  regarding 
productivity  measurement  using  measurements  prior  to  an 
automation  implementation  and  after. 

2.  There  is  little  documentation  regarding  the  measurement 
of  productivity  improvement  due  to  the  automation  of 
office  work. 

3.  The  use  of  job  satisfaction  surveys  as  an  indicator  of 
productivity  rather  than  input/output. 

4.  Although  there  is  still  a  belief  among  many  that 
computerization  of  office  work  leads  to  productivity 
enhancement,  there  are  numerous  examples  which  indicate 
that  this  view  is  changing  towards  one  requiring  actual 
justification. 
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Q .   OBJECTIVE 

The  preceeding  methodologies  are  based  on  a  common 
mechanism  for  the  measurement  of  productivity,  which  they 
define  as  a  ratio  of  outputs  to  inputs  in  one  form  or 
another.   This  study  will  also  use  this  ratio  model  in  the 
calculation  of  productivity. 

This  thesis  will: 


1 .  Develop  a  methodology  for  productivity  measurement  that 
is  based  on  a  before/after  quasi-experimental  design. 

2 .  Conduct  an  office  automation  study  to  test  the 
methodology . 

3.  Base  the  study  on  quantifiable  measures  of  input  and 
output . 
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III.    METHODOLOGY 

A.   CONDUCT  OF  THE  STUDY 

1 .  Prelude  to  the  Sample 

The  investigators  sampled  data  from  a  military 
procurement  organization:  the  Purchasing  Department  at  the 
Naval  Supply  Center  (NSC)  Duarte,  South  Dakota.   The 
selection  of  this  site  is  due  to  its  having  an  operational 
APADE  installation  and  the  availability  of  pre-APADE 
records.   This  study  has  provided  an  opportunity  to  measure 
productivity  changes  since  the  implementation  of  APADE  at 
NSC  Duarte. 

After  initial  discussions  with  the  staff,  it  was 
determined  that  the  primary  data  needed  to  conduct  this 
study  would  be  available.   We  decided  on  a  'before  and 
after'  data  collection  design  to  compare  productivity 
statistics  prior  to  and  after  the  implementation  of  APADE. 
Data  sampled  prior  to  the  implementation  of  APADE  were 
actual  purchase  orders  and  reports  from  the  on-site 
archives.   Data  for  after  implementation  was  acquired  from 
on-site  reports  as  well  as  documentation  generated  by  APADE. 
2  .  A  Description  of  APADE 

The  APADE  configuration  studied  is  a  small  purchase, 
computer-based  system  designed  to  provide  a  Navy  field 
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contracting  office  with  a  standard  automated  procurement 
mechanism.   It's  capabilities  include: 

1.  Automated  documentation  preparation. 

2.  Automated  document  control  and  tracking  status. 

3.  Procurement  management  information. 

4.  Buyer  support  information. 

This  system  provides  real-time  contracting 
information  to  management  while  allowing  selective  access  by 
customers.   Additional  information  on  APADE  is  presented  in 
Appendix  B. 

3.  Experimental  Design  Development 

Our  study  employed  a  quasi-experimental  methodology 
(Campbell  and  Stanley,  1968) .   Samples  were  taken  for  a 
twelve  month  period  before  and  a  twelve  month  period  after 
APADE  implementation.   It  was  determined  that  three  types  of 
data  would  be  used  to  evaluate  productivity: 

1.  Inputs. 

2.  Outputs. 

3.  Social  Effects. 

a.   Inputs 

These  include  the  measurement  of  quantifiable 
elements,  such  as  grade  structure,  size  of  staff,  and 
overtime  worked. 


22 


Jb.  Outputs 

Various  factors  that  are  the  product  of  work  done 
include  Procurement  Action  Lead  Time  (PALT) ,  number  of 
purchase  orders  processed  per  unit  time,  and  the  number  of 
labor  hours  per  purchase  order.   PALT  is  the  time  it  takes 
to  process  a  purchase  order.   The  PALT  counter  begins  upon 
the  receipt  of  a  purchase  action  request  by  the  Small 
Purchase  Branch  and  stops  when  a  purchase  order  has  been 
awarded.   The  number  of  purchase  orders  processed  during  a 
year  is  used  to  indicate  the  volume  of  work,  while  the 
number  of  labor  hours  needed  to  perform  a  single  procurement 
is  a  measure  of  productivity. 
c.  Socxa.1  Effects 

Social  effects  act  as  surrogate  indicators  to  the 
work  environment.  This  study  used  annual  leave,  sick  leave, 
and  leave  without  pay . 

The  amount  of  annual  leave  used  is  determined  by 
management's  capability  to  allow  personnel  to  expend  leave. 
If  workload  is  at  a  level  to  where  authorizing  leave  is  not 
detrimental  to  the  organization,  leave  may  be  granted. 
Conversely,  if  there  are  significant  backlogs  in  the 
workload,  management  may  not  have  the  flexibility  to  allow 
personnel  to  take  leave  on  a  discretionary  basis. 

Sick  leave  policy  provides  a  way  for  personnel 
who  are  unhappy  or  stressed  to  avoid  the  demands  of  the  work 
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place.   Sick  leave  is  earned  with  the  decision  to  use  it 
made  exclusively  by  the  employee.   Since  no  documentation  is 
required  from  a  physician  to  confirm  an  employee's  status, 
it  is  an  available  escape  mechanism  for  an  employee  who  is 
dissatisfied  with  the  work  environment  or  workload. 

Leave  Without  Pay  (LWOP)  is  similar  to  annual 
leave.   The  exception  is  that  employees  who  use  LWOP  do  not 
receive  any  monetary  compensation  while  in  a  LWOP  status. 
An  employee  would  use  LWOP  as  a  last  resort  when  they  do  not 
have  annual  leave  or  sick  leave  time  "on  the  books." 
4 .  Analysis  Strategy 

The  statistical  test  selected  to  evaluate  the  data 
is  the  difference  of  means  in  a  "Student's  t-distribution" 
test.   A  series  of  null  hypotheses  are  tested  for 
rejection/inability  to  reject,  using  the  t-test  for 
statistical  significance. 
5  .  Collection  of  Data 

Pre-APADE  data  were  collected  from  records 
maintained  in  storage  for  the  fiscal  year  1987.   The  data, 
obtained  by  manually  sampling  the  physical  records,  were 
used  to  establish  the  baseline  for  the  activity's  PALT. 
PALT  data  for  the  period  after  APADE  installation  were 
obtained  by  sampling  listings  generated  by  the  system. 
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B.   THE  COLLECTION  DESIGN 

The  Regional  Contracting  Department  is  made  up  of  two 
divisions,  which  in  turn  are  made  up  of  various  branches. 
This  study  concerns  itself  with  the  data  obtained  for  the 
Small  Purchase  Branch,  which  is  currently  staffed  with  58 
people,  primarily  female  civilian  employees. 

The  collection  of  data  from  the  Contracting  Department 
archives  involved  sampling  purchase  orders  before  and  after 
APADE  installation.   The  following  describes  how  records 
used  for  estimating  PALT  were  sampled. 

1 .  Before  APADE  Installation 

The  population  consisted  of  85479  records  for  fiscal 
year  1987  purchase  orders.   This  population  figure  was 
obtained  from  the  Monthly  Procurement  Summary  of  Actions 
Report  (DD  Form  1057) .   These  transactions  were  made  up  of 
either  a  single  requisition  or  multiple  requisitions  for 
similar  material  grouped  together  from  a  single  customer. 

In  sampling  the  purchase  orders,  we  randomly 
selected  records  from  departmental  archives.   A  total  of  532 
purchase  orders  were  sampled  from  the  period  prior  to  APADE 
being  implemented. 

2 .  After  APADE  Installation 

The  process  for  sampling  post  implementation 
purchase  orders  is  the  same  except  that  the  samples  were 
taken  from  listings  generated  by  APADE.   Our  population 
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consisted  of  59905  records  for  fiscal  year  1989  purchase 
orders  obtained  from  the  DD  Form  1057.   A  total  of  637 
purchase  orders  were  sampled  from  the  period  after  APADE  had 
been  implemented. 

3 .  Additional  Measures 

Our  study  also  required  that  we  retrieve  archival 
data  from  other  sources  as  well.   Data  on  the  use  of  annual 
leave,  sick  leave,  leave  without  pay,  and  overtime  hours 
worked  was  obtained  from  NSC  Duarte  Comptroller  Department's 
records  and  reports.   Information  regarding  the  number  of 
personnel  employed  and  grade  structure  within  the  Purchasing 
Division  was  also  obtained  from  the  command' s  Comptroller 
Department . 

A  framework  was  used  to  represent  the  interaction  of 
inputs  and  outputs,  prior  to  and  after  the  implementation  of 
APADE  at  NSC  Duarte.   (See  Table  I)   It  provides  a  means  to 
test  the  null  hypothesis  for  the  measures  listed.   In 
testing  a  null  hypothesis,  we  are  attempting  to  determine  if 
the  data  sampled  before  and  after  this  phase  of  APADE 
implementation  were  statistically  the  same.   The  alternative 
hypothesis  is  that  the  before  and  after  data  are  different. 

C .   INSTRUMENTATION 

One  method  to  measure  a  change  in  productivity  is  by 
analyzing  the  ratio  of  inputs  to  outputs.   In  a  system,  a 
certain  amount  of  input  is  required  to  produce  some  output. 
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TABLE  I 


PRODUCTIVITY  MATRIX 


MEASURES 

PRE  APADE 

APADE 

INPUTS 

Staff  Size 
Grade  Structure 
Overtime 

Staff  Size 
Grade  Structure 
Overtime 

OUTPUTS 

No.  Purchase 

Orders 

PALT 

No.  Purchase 

Orders 

PALT 

SOCIAL 
EFFECTS 

Annual  Leave 

Sick  Leave 

Leave  Without  Pay 

Annual  Leave 

Sick  Leave 

Leave  Without  Pay 

PRODUCTIVITY 
MEASUREMENT 

Purchase  Orders 
per  Labor  Hour 

Purchase  Orders 
per  Labor  Hour 

If  the  system  is  changed  so  as  to  require  fewer  inputs  or  to 
produce  more  output  with  the  same  inputs,  then  productivity 
is  enhanced. 
1 .  Input 8 

There  are  several  inputs  to  the  requisitioning 
process  that  were  used  in  this  evaluation: 

1.  Before  and  after  mean  annual  number  of  employees. 

2.  Before  and  after  mean  GS   level. 

3.  Before  and  after  mean  overtime  worked. 
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The  measurement  of  the  mean  annual  number  of 
employees  provided  a  figure  of  the  average  number  of 
personnel  required  by  the  Small  Purchase  Branch  to  perform 
its  function.   The  mean  GS  level  of  the  staff  provided  an 
indicator  of  the  level  of  expertise  to  perform  the  functions 
of  the  unit.   The  mean  overtime  worked  represented  the  extra 
time  spent,  beyond  regular  working  hours,  to  complete  the 
work  assigned. 
2 .  Outputs 

The  output  of  the  procurement  process  contained  two 
aspects  which  were  measured.   The  first  considered  the 
difference  between  the  number  of  requisitions  processed 
prior  to  and  after  implementation.   The  second  measured  the 
PALT  both  before  and  after  implementation. 

PALT  has  been  selected  as  our  Productivity  Measure 
of  Effectiveness,  because  it  provides  a  measure  of 
productivity.   PALT  represents  the  time  it  takes  to  process 
a  purchase  order  once  a  requisition  has  been  received  by  the 
purchasing  division.   A  typical  scenario  would  begin  with 
the  submission  of  a  requisition.   After  proceeding  through 
the  Customer  Service  and  Technical  Branches,  the  requisition 
is  determined  to  be  a  purchase  item  rather  than  something 
that  is  available  through  the  supply  system.   The  PALT  timer 
begins  when  a  requisition  is  received  by  the  Purchase 
Division.   Once  the  processing  has  been  completed  and  the 
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requisition  has  been  awarded  to  a  vendor,  the  PALT  timer 
stops.   The  difference  between  the  award  date  and  the  date 
of  receipt  of  the  requisition,  plus  one  day,  is  the  PALT  for 
a  purchase.2 

The  quantity  parameter  considered  the  volume  of  work 
performed  by  the  Small  Purchase  Branch  over  a  unit  of  time. 
In  measuring  quantity  we  used  the  average  number  of  purchase 
orders  processed  each  month. 
3 .  Social  Effects 

In  previous  studies  which  performed  similar  analyses 
(Barclift  and  Linson,  1988)  it  was  noted  that  various  social 
factors  affected  unit  performance.   Reduced  workload, 
increased  teamwork,  and  improvements  in  training  were 
indicators  that  were  mentioned.   The  social  effects  that 
were  measured  include: 

1.  Before  and  after  mean  annual  leave  taken. 

2.  Before  and  after  mean  sick  leave  taken. 

3.  Before  and  after  mean  leave  without  pay  used. 
Although  these  social  factors  are  not  directly  measures  of 
input  or  output,  they  are  objective  indicators  of 
satisfaction  obtained  from  the  working  environment. 


20ne  day  is  added  so  that  PALT  would  reflect  the  inclusive 
dates  of  receipt  of  the  requisition  and  the  award  date  of  the 
contract . 
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4 .  Productivity  Measurement 

The  productivity  measurement  used  in  this  study  is 
simply  the  number  of  transactions  processed/completed  per 
labor  hour.   Labor  hours  were  calculated  as  the  total  number 
of  hours  worked  by  both  purchasing  personnel  (buyers)  and 
administrative  support  personnel.   This  factor  provided  us 
with  a  direct  measurement  of  the  productivity  of  the 
purchasing  and  administrative  support  personnel. 

D.   ANALYSIS  STRATEGY 
1 .  Measurements 

A  difference  of  means  test  was  used  to  compare  the 
various  input ,  output,  social  factors,  and  productivity 
ratio  prior  to  and  after  the  implement at i or  of  APADE.   Null 
hypotheses  stated  that  there  was  no  change  in  the  various 
measurements  as  a  result  of  APADE' s  implementation. 

A  total  of  1169  purchase  orders  were  sampled.   Of 
these,  532  records  represented  the  PALT  prior  to 
implementation,  while  637  records  were  collected  for  the 
post  implementation  period. 

The  command' s  Comptroller  Department  provided  access 
to  staff  size,  grade  and  step  level,  annual  leave  taken, 
sick  leave  used,  and  leave  taken  without  pay  data  from  on- 
hand  NAVCOMPT  FORM  206  records  for  both  periods.   Staff  size 
was  verified  via  the  small  purchase  department's  point  of 
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contact.   Overtime  data  were  also  obtained  from  the 
command's  comptroller  departmental  records. 

2 .  Choosing  the  Appropriate  Statistical  Test 

When  testing  hypotheses  using  two  populations,  with 
the  samples  being  either  dependent  or  independent  of  one 
another,  and  having  an  unknown  population  standard  deviation 
(<7)  ,  the  appropriate  method  would  be  a  student's  t- 
distribution   (Porter  and  Hamm,  1986) .   We  used  the  t-test 
to  assess  differences  between  the  means  of  various 
indicators,  both  before  and  after  APADE  implementation,  for 
samples  taken  independently  from  the  two  populations. 

The  reason  for  taking  independent  samples  from  each 
population  is  that  the  pool  of  employees  within  the  division 
was  not  identical  before  and  after  implementation.   There 
was  the  usual  turnover  of  personnel  that  organizations 
endure  and  a  restructuring  of  the  unit  because  of  APADE. 

The  null  hypothesis  (H0)  stated  that  the  two 
populations  are  statistically  the  same,  while  the 
alternative  hypothesis  (H.)  stated  that  the  before  measure 
is  greater  or  less  than  the  after  measure  to  a  statistically 
significant  degree. 

Using  MINITAB  Fundamental /RW  Software  for  Students 
(Anderson/Eynon) ,  a  one  tailed  t-distribution  test  (TTEST) 
was  selected  because  we  were  predicting  direction.   The 
corresponding  significance  levels  are  shown  in  Table  II.   We 
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have  chosen  the  .01  significance  level  as  the  appropriate 
level  at  which  to  test  all  measures  collected. 


TABLE  II.   T-DISTRIBUTION  SIGNIFICANCE  LEVELS 


Significance  Level 

t-value 

.05 

1.645 

.01 

2.326 
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IV.    FINDINGS 

A.   INPUTS 

Measures  of  inputs  to  the  Small  Purchase  Branch  of  the 
Regional  Contracting  Department  located  at  NSC  Duarte,  were 
collected  and  summarized  in  the  ensuing  categories: 

1.  Size  of  the  staff. 

2.  Grade  structure  (GS  level). 

3.  Overtime  worked. 

These  factors  were  collected  for  time  periods  before  and 
after  the  implementation  of  APADE. 
1 .  Before  APADE 

a.     Staff  Size 

The  size  of  the  staff  was  calculated  as  the  mean 
size  of  the  staff  on  hand  for  each  pay  period  at  the  Small 
Purchasing  Branch  (SPB) .   The  mean  size  of  the  staff  before 
the  implementation  of  APADE  was  73.4  personnel  with  a 
standard  deviation  of  3.5.   This  represents  the  period  from 
January  1,  1987  through  December  31 ,  1987.   Data  were 
collected  from  NAVCOMPT  FORM  206' s  maintained  by  the 
command's  comptroller  department. 

Jb.  Grade   Structure 

The  grade  level  of  the  staff  was  computed  as  the 
average  GS  level  for  all  personnel  at  the  Small  Purchasing 
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Branch  (SPB)  per  pay  period.   The  mean  grade  level  (GS)  of 
the  staff  per  pay  period  was  GS  5.66,  with  a  standard 
deviation  of  0.098.   This  represents  the  period  from  January 
1,  1987  through  December  31,  1987.   Data  were  obtained  from 
worker's  Service  Card  (NAVCOMPT  FORM  206)  maintained  by  the 
command's  comptroller  department. 

c .  Overtime 

Overtime  worked  by  SPB  personnel  was  calculated 
as  the  mean  number  of  overtime  hours  worked  by  the  entire 
staff  for  each  two-week  pay  period  for  the  year  prior  to  the 
implementation  of  APADE.   The  average  overtime  worked  per 
two-week  pay  period  was  519.5  hours,  with  a  standard 
deviation  of  306.0.   The  mean  overtime  worked  per  worker  per 
two-week  pay  period  was  6.7  hours,  with  a  standard  deviation 
of  7.1.   The  mean  overtime  worked  per  buyer  in  the  SPB  was 
177.3  hours  annually,  with  a  standard  deviation  of  145.6. 
The  mean  overtime  worked  per  administrative  support  worker 
was  128.9  hours,  with  a  standard  deviation  of  121.6.   The 
annual  mean  overtime  worked  by  all  SPB  personnel  was  162.7 
hours  per  worker,  with  a  standard  deviation  of  139.9. 
2 .  After  APADE  Implementation 

a.  Staff  Size 

The  mean  size  of  the  staff  after  the 
implementation  of  APADE  was  62.4  personnel,  with  a  standard 
deviation  of  1.24.   This  represents  the  period  from  January 
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1,  1989  through  December  31,  1989.   Data  were  collected  from 
NAVCOMPT  FORM  206' s  maintained  by  the  command's  comptroller 
department . 

b.  Grade   Structure 

The  mean  grade  level  (GS)  of  the  staff  per  pay 
period  was  GS  6.06,  with  a  standard  deviation  of  0.006. 
This  represents  the  period  from  January  1,  198  9  through 
December  31,  1989.   Data  were  obtained  from  worker's  Service 
Card  (NAVCOMPT  FORM  206)  maintained  by  the  command's 
comptroller  department. 

c.  Overtime 

The  average  overtime  worked  per  two-week  pay 
period  was  124.4  hours,  with  a  standard  deviation  of  168.9. 
The  mean  overtime  worked  per  worker  per  two-week  pay  period 
was  2.0  hours,  with  a  standard  deviation  of  2.7.   The 
average  amount  of  overtime  worked  per  buyer  in  the  SPB  was 
58.1  hours  annually,  with  a  standard  deviation  of  71.7.   The 
average  amount  of  overtime  worked  annually  per 
administrative  support  worker  was  25.2  hours,  with  a 
standard  deviation  of  33.5.   The  average  amount  of  overtime 
worked  annually  by  all  SPB  personnel  was  51.3  hours  per 
worker  with  a  standard  deviation  of  66.8. 
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3 .  Testing  the  Null  Hypotheses  for  Inputs 
a.  Staff  Size 

The  null  hypothesis  (H0)  concerning  the  size  of 
the  staff  at  the  Small  Purchasing  Branch  (SPB)  stated  that 
the  size  of  the  staff  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  was  statistically  equal  to  that 
prior  to  implementation  (X0)  ,  (H0:  X=X0)  .   The  alternative 
hypothesis  (H.)  stated  that  the  size  of  the  staff  per  two- 
week  pay  period  after  the  implementation  (X)  is  less  than 
the  size  prior  to  the  implementation  (X0)  ,  (H. :  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -44.97  at  the  0.0000  significance  level 
(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (Ha)  was 
accepted;  mean  staff  size  at  the  SPB  per  two-week  pay  period 
after  the  implementation  of  APADE  (X)  was  statistically  less 
than  the  mean  staff  size  prior  to  implementation  (X0)  . 

Table  III  represents  the  number  of  buyers  and 
administrative  support  personnel  immediately  before  the 
implementation  of  APADE  and  at  the  end  of  first  full  year  of 
APADE  operation. 
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TABLE  III.  EMPLOYEES 


PRE  APADE 

APADE 

BUYERS 

58 

50 

ADMINISTRATIVE 
SUPPORT 

25 

13 

b.     Grade  Structure 

The  null  hypothesis  (H0)  of  the  grade  structure 
of  the  staff  at  the  SPB  stated  that  the  grade  level  of  the 
staff  per  two-week  pay  period  after  the  implementation  (X) 
of  APADE  was  statistically  equal  to  that  prior  to 
implementation  (X0)  :   (Ho:  X=X0)  .   The  alternative 
hypothesis  (Ha)  stated  that  the  grade  level  of  the  staff  per 
two-week  pay  period  after  the  implementation  (X)  is  greater 
than  the  level  prior  to  the  implementation  (X0)  ,  (Ha:  X>X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  370.15  at  the  0.0000  significance  level 
(right-tailed  test) .   Since  this  is  greater  than  the  t- 
distribution  baseline  for  this  study  (t-value  2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (H.)  was 
accepted;  mean  grade  level  at  the  SPB  per  two-week  pay 
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period  after  the  implementation  of  APADE  (X)  was 
statistically  greater  than  the  mean  staff  level  prior  to 
implementation  (X0)  . 
c.  Overtime 

The  first  null  hypothesis  (H0)  concerning 
overtime  stated  that  the  overtime  worked  by  all  personnel 
per  two-week  pay  period  after  the  implementation  of  APADE 

(X)  is  statistically  equal  to  that  worked  prior  to 
implementation  (X0)  ,  (H0:  X=X0)  .   The  alternative  hypothesis 

(H„)  stated  that  the  overtime  worked  by  all  personnel  per 
two-week  pay  period  after  the  implementation  (X)  is  less 
than  that  worked  prior  to  the  implementation  (X0)  ,  (Ha: 
X<X0)  . 

Analysis  finds  that  the  t-distribution  test  score 
for  this  measure  is  -11.93  at  the  0.0000  significance  level 

(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value 
-2.326  at  the  0.01  significance  level),  the  null  hypothesis 

(H0)  was  rejected.   Therefore,  the  alternative  hypothesis 

(H.)  was  accepted;  mean  overtime  worked  by  all  personnel  per 
two-week  pay  period  after  the  implementation  of  APADE  (X) 
was  statistically  less  than  the  mean  overtime  worked  prior 
to  implementation  (Xc)  . 

The  second  null  hypothesis  (H0)  for  overtime 
worked  stated  that  the  overtime  worked  per  worker  per  two- 
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week  pay  period  after  the  implementation  of  APADE  (X)  is 
statistically  equal  to  that  worked  prior  to  implementation 

(X0)  ,  (H0:  X=X0)  .   The  alternative  hypothesis  (H.)  stated 
that  the  overtime  worked  per  worker  per  two-week  pay  period 
after  the  implementation  (X)  is  less  than  that  worked  prior 
to  the  implementation  (X0)  ,  (Ha:  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -14.06  at  the  0.0000  significance  level 

(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (Ha)  was 
accepted;  mean  overtime  worked  per  worker  per  two-week  pay 
period  after  the  implementation  of  APADE  (X)  was 
statistically  less  than  the  mean  overtime  worked  prior  to 
implementation  (X0)  . 

Tables  IV  and  V  represent  a  summary  of  the  total 
amount  of  overtime  worked  before  and  after  implementation. 
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TABLE  IV. 


PRE  APADE  OVERTIME 


TYPE 
PERSONNEL 

OVERTIME 

NO.  OF 
PERIODS 

MEAN 
OVERTIME 

BUYERS 

10285.0 

26 

177.3 

ADMIN 

3221.3 

26 

128.9 

TOTAL 

13506.3 

26 

162.7 

TABLE  V. 


APADE  OVERTIME 


TYPE 
PERSONNEL 

OVERTIME 

NO.  OF 
PERIODS 

MEAN 
OVERTIME 

BUYERS 

2906.2 

26 

58.1 

ADMIN 

327.8 

26 

25.2 

TOTAL 

3234.0 

26 

51.3 

B .   OUTPUTS 

Measures  of  outputs  from  the  Small  Purchase  Branch  of 
the  Regional  Contracting  Department  at  NSC  Duarte  were 
collected  and  summarized  in  the  ensuing  categories: 

1.  Number  of  purchase  orders. 

2.  Procurement  Action  Lead  Time  (PALT) . 

These  factors  were  collected  for  time  periods  before  and 
after  the  implementation  of  APADE. 
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1.   Before  APADE 

a.  Number  of  Purchase   Orders 

The  number  of  purchase  requests  processed  is 
calculated  as  the  number  of  purchase  orders  completed  for 
each  pay  period  for  the  year  prior  to  the  implementation  of 
APADE.   The  average  number  of  purchase  requests  processed 
per  pay  period  by  the  SPB  before  the  implementation  of  APADE 
was  3287.7,  with  a  standard  deviation  of  655.1.   This 
represents  both  single  and  multiple  item  procurements  for 
fiscal  year  1987.   Data  were  collected  from  the  Procurement 
Summary  of  Actions  Report  (DD  FORM  1057)  that  was  manually 
generated  on  site. 

b.  Procurement  Action  Lead   Time    (PALT) 

PALT  is  computed  as  the  average  number  of  days 
required  to  complete  a  purchase  request,  from  the  date 
received  by  the  SPB  to  the  date  the  contract  for  the 
purchase  request  is  awarded  to  a  contractor,  plus  one  day.3 
PALT  prior  to  the  implementation  of  APADE  was  52.4  days, 
with  a  standard  deviation  of  33.8.   Data  were  collected  from 
a  sampling  of  the  actual  purchase  order  records  that  were 
retained  on  site.   See  Table  VI  for  a  summary  of  the  samples 
taken  prior  to  the  implementation  of  APADE. 


3The  one  day  is  added  so  that  PALT  will  represent  the  number 
of  days  between  the  receipt  date  and  award  date,  inclusive. 
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2 .  After  APADE  Implementation 

a.  Number  of  Purchase   Orders 

The  average  number  of  purchase  requests  processed 
per  pay  period  by  the  SPB  after  the  implementation  of  APADE 
was  2304.0,  with  a  standard  deviation  of  454.4.   This 
represents  both  single  and  multiple  item  procurements  for 
fiscal  year  198  9  (after  the  implementation  of  APADE) .   Data 
were  collected  from  the   Procurement  Summary  of  Actions 
Report  (DD  FORM  1057)  that  was  manually  generated  on  site. 

b.  Procurement  Action  Lead  Time    (PALT) 

PALT  after  the  implementation  of  APADE  was  23.5 
days,  with  a  standard  deviation  of  24.7.   Data  were 
collected  from  a  sampling  of  the  actual  purchase  order 
records  that  were  retained  on  site.   See  Table  VI  for  a 
summary  of  the  samples  taken  after  the  implementation  of 
APADE. 

TABLE  VI.    SAMPLE  BREAKDOWN  BY  ISSUE  GROUP 


PERIOD 

ISSUE  GROUP 
1 

ISSUE  GROUP 
2 

ISSUE  GROUP 
3 

PRE  APADE 
(FY  1987) 

69 
12.97% 

260 
48.87% 

203 
38.16% 

APADE 
(FY  1989) 

118 
18.52% 

333 
52.28% 

186 
29.20% 
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3 .  Testing  the  Null  Hypotheses  for  Outputs 

a.  Number  of  Purchase   Orders 

The  null  hypothesis  (H0)  concerning  the  number  of 
purchase  orders  stated  that  the  mean  number  of  purchase 
orders  processed  per  pay  period  after  the  implementation  of 
APADE  (X)  is  statistically  equal  to  the  number  processed  per 
pay  period  prior  to  implementation  (X0)  ,  (H0:  X=X0)  .   The 
alternative  hypothesis  (Ha)  stated  that  the  mean  number  of 
purchase  orders  processed  per  pay  period  (X)  is  less  than 
the  number  processed  per  pay  period  prior  to  implementation 

(X0),  (H.:  X<XJ  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -11.04  at  the  0.0000  significance  level 

(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value 
-2.326  at  the  0.01  significance  level),  the  null  hypothesis 
was  rejected.   Therefore,  the  alternative  hypothesis  (H.) 
was  accepted;  the  mean  number  of  purchase  orders  processed 
per  pay  period  after  the  implementation  of  APADE  (X)  was 
statistically  less  than  that  prior  to  implementation  (X0)  . 

b.  Procurement  Action  Lead  Time    (PALT) 

The  null  hypothesis  (H0)  concerning  the  average 
PALT  stated  that  the  mean  value  obtained  after  the 
implementation  of  APADE  (X)  is  statistically  equal  to  the 
value  obtained  prior  to  implementation  (X0)  ,  (He:  X=X0)  .   The 


43 


alternative  hypothesis  (Ha)  stated  that  the  mean  PALT  value 
(X)  is  less  than  the  value  obtained  prior  to  implementation 
(X0),  (H.:  X<XJ  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -29.49  at  the  0.0000  significance  level 
(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (H.)  was 
accepted;  the  mean  PALT  after  the  implementation  of  APADE 
(X)  was  statistically  less  than  that  prior  to  implementation 
(X0) .   Tables  VII  and  VIII  represent  the  summary  of 
Procurement  Action  Lead  Time  (PALT)  for  the  before  and  after 
periods . 


TABLE  VII.    PALT  SUMMARY  BY  ISSUE  GROUP 


PERIOD 

SAMPLE  MEAN 

STANDARD 
DEVIATION 

IG 

1 

IG 
2 

IG 
3 

IG 

1 

IG 
2 

IG 
3 

PRE  APADE 
(FY  1987) 

48.07 

55.72 

50.21 

35.85 

35.97 

30.62 

APADE 
(FY  1989) 

11.20 

23.98 

30.52 

18.29 

24.11 

26.44 
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TABLE  VIII. 


PALT  SUMMARY  BY  PERIOD 


PERIOD 

SAMPLE 
MEAN 

STANDARD 
DEVIATION 

PRE  APADE 
(FY-1987) 

52.398 

33.792 

APADE 
(FY-1989) 

23.52 

24.713 

C.   SOCIAL  EFFECTS 

Measures  of  social  effects  at  the  Small  Purchase  Branch 
(SPB)  of  the  Regional  Contracting  Department  at  NSC  Duarte, 
were  collected  and  summarized  in  the  ensuing  categories: 

1.  Amount  of  annual  leave  taken. 

2.  Amount  of  sick  leave  used. 

3.  Amount  of  leave  without  pay  utilized. 

These  factors  were  collected  for  time  periods  before  and 
after  the  implementation  of  APADE. 
1.  Before  APADE 

a.  Annual  Leave 

The  amount  of  annual  leave  used  by  SPB  personnel 
was  calculated  as  the  mean  number  of  hours  used  by  all 
buyers  and  administrative  support  personnel  for  each  two- 
week  pay  period  for  the  year  before  the  implementation  of 
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APADE.   The  average  amount  of  annual  leave  used  per  two-week 
pay  period  for  the  SPB  staff  was  442.2  hours,  with  a 
standard  deviation  of  202.5.   The  average  amount  of  annual 
leave  used  per  worker  per  two-week  pay  period  was  5.4  hours, 
with  a  standard  deviation  of  2.2.   The  average  amount  of 
annual  leave  used  per  buyer  in  the  SPB  was  152.9  hours 
annually,  with  a  standard  deviation  of  52.0.   The  average 
amount  of  annual  leave  used  per  administrative  support 
worker  was  105.1  hours  annually,  with  a  standard  deviation 
of  50.9.   The  amount  of  annual  leave  used  by  all  SPB 
personnel  was  138.5  hours  per  worker  annually,  with  a 
standard  deviation  of  55.9.   See  Table  IX  for  a  summary 
breakdown  by  personnel  type  for  the  period  prior  to 
implementation  of  APADE. 
b.      Sick   Leave 

The  amount  of  sick  leave  used  by  SPB  personnel 
was  calculated  as  the  mean  number  of  hours  used  by  all 
buyers  and  administrative  support  personnel  for  each  two- 
week  pay  period  for  the  year  before  the  implementation  of 
APADE.   The  average  amount  of  sick  leave  used  per  two-week 
pay  period  for  the  SPB  staff  was  255.8  hours,  with  a 
standard  deviation  of  69.5.   The  average  amount  of  sick 
leave  used  per  worker  per  two-week  pay  period  was  3.2  hours, 
with  a  standard  deviation  of  1.5.   The  average  amount  of 
sick  leave  used  annually  per  buyer  in  the  SPB  was  82.2 
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hours,  with  a  standard  deviation  of  36.4.   The  average 
amount  used  per  administrative  support  worker  75.3  hours 
annually,  with  a  standard  deviation  of  4  6.5.   The  average 
amount  of  sick  leave  used  by  all  SPB  personnel  was  80.1 
hours  per  worker  annually,  with  a  standard  deviation  of 
39.6.   See  Table  IX  for  a  summary  breakdown  by  personnel 
type  for  the  period  prior  to  implementation  of  APADE. 


TABLE  IX. 


PRE  APADE  SOCIAL  EFFECTS 


TYPE 

ANNUAL 
LEAVE 

SICK 
LEAVE 

LEAVE 
WITHOUT 
PAY 
(LWOP) 

MEAN 

ANNUAL 

LEAVE 

MEAN 
SICK 
LEAVE 

MEAN 
LWOP 

BUYERS 

8870 

4768 

3243 

152.9 

82.2 

55.9 

ADMIN 

2628 

1883 

796 

105.1 

75.3 

31.9 

OVERALL 

11498 

6651 

4039 

138.5 

80.1 

48.7 

c.     Leave   Without  Pay 

The  amount  of  leave  without  pay  used  by  SPB 
personnel  was  calculated  as  the  mean  number  of  hours  used  by 
all  buyers  and  administrative  support  personnel  for  each 
two-week  pay  period  for  the  year  before  the  implementation 
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of  APADE.   The  average  amount  of  leave  without  pay  used  per 
two-week  pay  period  for  the  SPB  staff  was  155.4  hours,  with 
a  standard  deviation  of  77.3.   The  average  amount  of  leave 
without  pay  used  per  worker  per  two-week  pay  period  was  1 . 9 
hours,  with  a  standard  deviation  of  5.6.   The  average  amount 
of  leave  without  pay  used  per  buyer  in  the  SPB  was  55.9 
hours  annually,  with  a  standard  deviation  of  171.0.   The 
average  amount  of  leave  without  pay  used  annually  per 
administrative  support  worker  was  31.9  hours,  with  a 
standard  deviation  of  51.0.   The  average  amount  of  leave 
without  pay  used  by  all  SPB  personnel  was  4  8.7  hours  per 
worker  annually,  with  a  standard  deviation  of  145.6.   See 
Table  IX  for  a  summary  breakdown  by  personnel  type  for  the 
period  prior  to  implementation  of  APADE. 
2 .  After  APADE  Implementation 
a.  Annual  Leave 

The  average  amount  of  annual  leave  used  per  two- 
week  pay  period  for  the  SPB  staff  was  380.1  hours,  with  a 
standard  deviation  of  152.5.   The  average  amount  of  annual 
leave  used  per  worker  per  two-week  pay  period  was  6.1  hours, 
with  a  standard  deviation  of  2.0.   The  average  amount  of 
annual  leave  used  per  buyer  in  the  SPB  was  158.4  hours 
annually,  with  a  standard  deviation  of  63.8.   The  average 
amount  of  annual  leave  used  per  administrative  support 
worker  was  151.0  hours  annually,  with  a  standard  deviation 
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of  27.9.   The  amount  of  annual  leave  used  by  all  SPB 
personnel  was  156.8  hours  per  worker  annually,  with  a 
standard  deviation  of  48.8.   See  Table  X  for  a  summary 
breakdown  by  personnel  type  for  the  period  after  the 
implementation  of  APADE . 
Jb .  Sick  Leave 

The  average  amount  of  sick  leave  used  per  two- 
week  pay  period  for  the  SPB  staff  was  262.0  hours,  with  a 
standard  deviation  of  78.3.   The  average  amount  of  sick 
leave  used  per  worker  per  two-week  pay  period  was  4.2  hours, 
with  a  standard  deviation  of  2.4.   The  average  amount  of 
sick  leave  used  annually  per  buyer  in  the  SPB  was  105.3 
hours,  with  a  standard  deviation  of  63.8.   The  average 
amount  used  per  administrative  support  worker  was  119.3 
hours  annually,  with  a  standard  deviation  of  54.2.   The 
average  amount  of  sick  leave  used  by  all  SPB  personnel  was 
108.2  hours  per  worker  annually,  with  a  standard  deviation 
of  61.8.   See  Table  X  for  a  summary  breakdown  by  personnel 
type  for  the  period  after  the  implementation  of  APADE. 
c.  Leave   Without   Pay 

The  average  amount  of  leave  without  pay  used  per 
two-week  pay  period  for  the  SPB  staff  was  112.1  hours,  with 
a  standard  deviation  of  65.5.   The  average  amount  of  leave 
without  pay  used  per  worker  per  two-week  pay  period  was  1 . 8 
hours,  with  a  standard  deviation  of  4.8.   The  average  amount 
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of  leave  without  pay  used  per  buyer  in  the  SPB  was  48.8 
hours  annually,  with  a  standard  deviation  of  136.0.   The 
average  amount  of  leave  without  pay  used  annually  per 
administrative  support  worker  was  36.6  hours,  with  a 
standard  deviation  of  52.1.   The  average  amount  of  leave 
without  pay  used  annually  by  all  SPB  personnel  was  4  6.3 
hours  per  worker,  with  a  standard  deviation  of  123.2.   See 
Table  X  for  a  summary  breakdown  by  personnel  type  for  the 
period  after  the  implementation  of  APADE. 


TABLE  X. 


APADE  SOCIAL  EFFECTS 


TYPE 
PERSONNEL 

ANNUAL 
LEAVE 

SICK 
LEAVE 

LEAVE 
WITHOUT 
PAY 
(LWOP) 

MEAN 

ANNUAL 

LEAVE 

MEAN 
SICK 
LEAVE 

MEAN 
LWOP 

BUYERS 

7918 

5262 

2439 

158.4 

105.3 

48.8 

ADMIN 

1963 

1551 

475 

151.0 

119.3 

36.6 

TOTAL 

9881 

6813 

2914 

156.8 

108.2 

46.3 
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3 .  Testing  the  Null  Hypothesis  for  Social  Effects 
a .  Annual  Leave 

The  first  null  hypothesis  (H0)  concerning  annual 
leave  stated  that  the  amount  of  leave  taken  by  all  workers 
per  two-week  pay  period  after  the  implementation  of  APADE 

(X)  is  statistically  equal  to  that  prior  to  implementation 

(X0)  :  (H0:  X=X0)  .   The  alternative  hypothesis  (Ha)  stated 
that  amount  of  leave  taken  by  all  workers  per  two-week  pay 
period  after  the  implementation  (X)  is  less  than  that  amount 
taken  prior  to  the  implementation  (X0)  ,     (Ha:  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -2.08  at  the  0.024  significance  level 

(left-tailed  test) .   Since  this  is  not  less  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  was  not 
rejected.   Therefore,  the  mean  annual  leave  taken  by  all 
personnel  per  two-week  pay  period  after  the  implementation 
of  APADE  (X)  was  statistically  equal  to  that  prior  to 
implementation . 

The  second  null  hypothesis  (H0)  concerning  annual 
leave  stated  that  the  amount  of  leave  taken  per  worker  per 
two-week  pay  period  after  the  implementation  of  APADE  (X)  is 
statistically  equal  to  that  taken  prior  to  implementation 

(X0)  ,  (H0:  X=X0)  .   The  alternative  hypothesis  (Ha)  stated 
that  the  amount  of  leave  taken  per  worker  per  two-week  pay 
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period  after  the  implementation  (X)  is  more  than  that  amount 
taken  prior  to  the  implementation  (X0)  ,  (Ha:  X>X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  2.90  at  the  0.0026  significance  level 

(right-tailed  test) .   Since  this  is  greater  than  the  t- 
distribution  baseline  for  this  study  (t-value  2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (Ha)  was 
accepted;  the  mean  annual  leave  taken  per  worker  per  two- 
week  pay  period  after  the  implementation  of  APADE  (X)  was 
statistically  greater  than  that  prior  to  implementation 

<XJ  . 

b.     Sick  Leave 

The  first  null  hypothesis  (H0)  concerning  sick 
leave  stated  that  the  mean  amount  of  sick  leave  taken  by  all 
workers  per  two-week  pay  period  after  the  implementation  of 
APADE  (X)  is  statistically  equal  to  that  prior  to 
implementation  (X0)  ,  (H0:  X=X0)  .   The  alternative  hypothesis 

(Ha)  stated  that  the  mean  amount  of  sick  leave  taken  by  all 
workers  per  two-week  pay  period  after  the  implementation  (X) 
is  greater  than  that  amount  taken  prior  to  the 
implementation  (X0)  ,  (Ha:  X>X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  0.40  at  the  0.34  significance  level  (right- 
tailed  test) .   Since  this  is  not  greater  than  the  t- 
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distribution  baseline  for  this  study  (t-value  2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  could  not 
be  rejected.   Therefore,  the  mean  amount  of  sick  leave  taken 
by  all  personnel  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  was  statistically  equal  to  that 
prior  to  implementation  (X0)  . 

The  second  null  hypothesis  (H0)  concerning  sick 
leave  stated  that  the  mean  amount  of  sick  leave  taken  per 
worker  per  two-week  pay  period  after  the  implementation  of 
APADE  (X)  is  statistically  equal  to  that  taken  prior  to 
implementation  (X0)  ,  (H0:  X=X0)  .   The  alternative  hypothesis 

(Ha)  stated  that  the  mean  amount  of  sick  leave  taken  per 
worker  per  two-week  pay  period  after  the  implementation  (X) 
is  more  than  that  amount  taken  prior  to  the  implementation 

<XJ,  (H.:  X>XJ  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  3.50  at  the  0.0009  significance  level 

(right-tailed  test) .   Since  this  is  greater  than  the  t- 
distribution  baseline  for  this  study  (t-value  2.32  6  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was 
rejected.   Therefore,  the  alternative  hypothesis  (H.)  was 
accepted;  the  mean  amount  of  sick  leave  taken  per  worker  per 
two-week  pay  period  after  the  implementation  of  APADE  (X) 
was  statistically  greater  than  that  prior  to  implementation 

(XJ  . 
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c.   .Leave  Without   Pay 

One  of  the  null  hypotheses  (H0)  concerning  leave 
without  pay  stated  that  the  mean  amount  of  leave  without  pay 
taken  by  all  workers  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  is  statistically  equal  to  that 
prior  to  implementation  (X0)  ,  (H0:  X=X0)  .   The  alternative 
hypothesis  (Ha)  stated  that  the  mean  amount  of  leave  without 
pay  taken  by  all  workers  per  two-week  pay  period  after  the 
implementation  (X)  is  less  than  that  amount  taken  prior  to 
the  implementation  (X0)  ,  (Ha:  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -3.37  at  the  0.001  significance  level 
(left-tailed  test) .   Since  this  is  less  than  the  t- 
distribution  baseline  for  this  study  (t-value 
-2.326  at  the  0.01  significance  level),  the  null  hypothesis 
(H0)  was  rejected.   Therefore,  the  alternative  hypothesis 
(H.)  was  accepted;  the  mean  amount  of  leave  without  pay 
taken  by  all  personnel  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  was  statistically  less  than  that 
prior  to  implementation  (X0)  . 

The  other  null  hypothesis  (H0)  concerning  leave 
without  pay  stated  that  the  mean  amount  of  leave  without  pay 
taken  per  worker  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  is  statistically  equal  to  that 
taken  prior  to  implementation  (X0)  ,  (He:  X=X0)  .   The 
alternative  hypothesis  (Ha)  stated  that  the  mean  amount  of 
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leave  without  pay  taken  per  worker  per  two-week  pay  period 
after  the  implementation  (X)  is  less  than  that  amount  taken 
prior  to  the  implementation  (X0)  ,  (Ha:  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -0.12  at  the  0.55  significance  level  (left- 
tailed  test) .   Since  this  is  not  less  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  (H0)  was  not 
rejected.   Therefore,  the  mean  amount  of  leave  without  pay 
taken  per  worker  per  two-week  pay  period  after  the 
implementation  of  APADE  (X)  was  statistically  equal  to  that 
prior  to  implementation  (Xc)  . 

D.   PRODUCTIVITY  MEASUREMENT 

1.  Before  APADE 

The  number  of  purchase  orders  processed  per  labor 
hour  is  the  measurement  of  the  number  of  purchase  orders 
processed  during  the  year  divided  by  the  number  of  actual 
labor  hours  accrued  during  the  year.   The  mean  number  of 
purchase  orders  processed  per  labor  hour  prior  to  the 
implementation  of  APADE  was  0.5564  purchase  orders  per  hour, 
with  a  standard  deviation  of  0.1019. 

2 .  After  APADE  Implementation 

The  mean  number  of  purchase  orders  processed  per 
labor  hour  after  the  implementation  of  APADE  was  .5236  per 
hour,  with  a  standard  deviation  of  .0882.   The  data  were 
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collected  from  both  the  Procurement  Summary  of  Actions 
Report  (DD  FORM  1057)  that  was  manually  generated  on  site 
and  the  SPB  personnel'  Service  Card  (NAVCOMPT  FORM  20  6  - 
MECHANIZED) . 

3 .  Testing  the  Null  Hypothesis  for  Productivity 

The  null  hypothesis  concerning  the  productivity 
measurement  at  the  SPB  stated  that  the  number  of  purchase 
orders  processed  per  labor  hour  after  the  implementation  of 
APADE  (X)  was  statistically  equal  to  that  prior  to 
implementation  (H0:  X=X0)  .   The  alternative  hypothesis 
stated  that  the  number  of  purchase  orders  processed  per 
labor  hour  after  the  implementation  is  less  than  that  prior 
to  the  implementation  (Ha:  X<X0)  . 

Upon  analysis,  the  t-distribution  test  score  for 
this  measure  was  -1.90  at  the  0.035  significance  level 
(left-tailed  test) .   Since  this  is  greater  than  the  t- 
distribution  baseline  for  this  study  (t-value  -2.326  at  the 
0.01  significance  level),  the  null  hypothesis  is  not 
rejected.   The  productivity  measurement  at  the  SPB  after  the 
implementation  of  APADE  (X)  was  statistically  equal  to  that 
prior  to  implementation. 

E.   SUMMARY  OF  FINDINGS 

A  summary  of  T-Test  results  of  the  inputs,  outputs, 
productivity  measurement,  and  social  effects  are  presented 
in  tables  XI,  XII,  XIII,  and  XIV  respectively. 
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TABLE  XI. 


SUMMARY  OF  T-TEST  RESULTS:  INPUTS 


INPUTS 

PRE-APADE 

APADE 

T-TEST 
RESULTS 

MEAN  STAFF  SIZE 
Std.  Deviation 

73.35 
3.48 

62.42 
1.24 

T  =  -44. 97 
p  b  0.0000 

GRADE  LEVEL 
Std.  Deviation 

5.66 
.098 

6.06 
.006 

T  =  370.15 
p  =  0.0000 

MEAN  OVERTIME 
(per  2-week  pay 
period)  Std.  Dev. 

519.46 
306.03 

124.383 
168.894 

T  =  -11.93 
p  =  0. 0000 

MEAN  OVERTIME 
(per  worker  per  2- 
week  period) 
Std.  Dev. 

6.7229 
7.0831 

2.0270 
2.6508 

T  =  -14.06 
p  «  0.0000 

TABLE  XII. 


SUMMARY  OF  T-TEST  RESULTS:  OUTPUTS 


OUTPUTS 

PRE-APADE 

APADE 

T-TEST 
RESULTS 

MEAN  PURCHASE 
REQUESTS  PROCESSED 

3287.7 

2304.0 

T  =  -11.04 
p  =  0.0000 

Std.  Dev. 

655.1 

454.4 

MEAN  PROCUREMENT 
ACTION  LEAD  TIME 

52.4 

23.5 

T  =  -29.49 
p  =  0.0000 

Std.  Dev. 

33.8 

24.7 
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TABLE  XIII 


PRODUCTIVITY  MEASUREMENT  T-TEST  RESULTS 


PRODUCTIVITY 
MEASUREMENT 

PRE-APADE 

APADE 

T-TEST 
RESULTS 

MEAN  PURCHASE 
REQUESTS  PROCESSED 
PER  LABOR  HOUR 

0.5564 

0.5236 

T  =  -1.90 
p  =  0.035 

Standard  Deviation 

0.1019 

0.0882 

TABLE  XIV. 


SUMMARY  OF  T-TEST  RESULTS:  SOCIAL  EFFECTS 


SOCIAL  EFFECTS 

PRE-APADE 

APADE 

T-TEST 
RESULTS 

MEAN  ANNUAL  LEAVE 
(per  2-week  pay 
period) 
Std.  Dev. 

442.22 
202.46 

380.05 
152.47 

T  =  -2.08 
p  «  0.024 

MEAN  ANNUAL  LEAVE 
(per  worker  per  2- 
week  period) 
Std.  Dev. 

5.3995 
2.1938 

6.1203 
1.9751 

T  =  2.90 
p  =  0.0026 

MEAN  SICK  LEAVE 
(per  2 -week  pay 
period) 
Std.  Dev. 

255.83 
69.459 

262.03 
78.313 

T  =  0.40 
p  «  0.34 

MEAN  SICK  LEAVE 
(per  worker  per  2- 
week  period) 
Std.  Dev. 

3.1539 
1.5326 

4.2018 
2.3732 

T  =  3.50 
p  =  0.0009 

MEAN  LEAVE  WITHOUT 
PAY  (per  2 -week 
pay  period) 
Std.  Dev. 

155.40 
77.29 

112.05 
65.536 

T  =  -3.37 
p  «  0.0012 

MEAN  LEAVE  WITHOUT 
PAY  (per  worker 
per  2-week  period) 
Std.  Dev. 

1.8717 
5.6010 

1.8003 
4.7991 

T  =  -0.12 
p  =  0.55 
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V.    ANALYSIS  AND  CONCLUSION 

A.   INPUTS 

1 .   Staff  Size 

The  mean  staff  size  prior  to  the  implementation  of 
APADE  was  73.4  personnel.   The  mean  staff  size  after 
implementation  was  62.4  personnel.   This  represents  a 
decrease  of  15  percent  in  the  number  of  personnel  in  the 
Small  Purchasing  Branch  (SPB)  at  NSC  Duarte .   With  the 
remaining  factors  being  held  constant,  a  decrease  in 
headcount  would  provide  a  decreased  input  measure.   Combined 
with  a  constant  output  measure,  by  definition,  using  the 
ratio  of  outputs  to  inputs,  this  by  itself  would  provide  an 
increase  in  productivity. 

The  amount  of  administrative  support  required  prior 
to  implementation  was  not  needed  after  APADE' s 
implementation.   This  is  due  to  APADE' s  design  allowing 
buyers  to  enter  the  data  into  the  system  as  events  occur 
rather  than  transporting  documents  to  data  entry  clerks  for 
input.   This  decreases  the  number  of  administrative  support 
personnel  required  for  support.   Although  a  decrease  in 
staff  size  by  itself  would  act  to  increase  productivity  the 
implementation  of  APADE  is  not  the  only  factor  which  may 
have  had  this  effect.   The  number  of  purchase  orders 
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submitted  to  NSC  Duarte  had  also  decreased.   Management 
sources  indicate  that  the  decrease  in  the  number  of  purchase 
requests  received  had  no  effect  on  the  decrease  in  the 
number  of  personnel  between  the  pre  and  post  APADE  periods. 
However,  because  regression  analysis  was  beyond  the  scope  of 
this  study  we  did  not  address  the  relative  significance  of 
these  two  factors.   Therefore,  it  can  not  be  categorically 
stated  that  the  decrease  in  staff  size  was  in  whole  or  part 
due  to  the  implementation  of  APADE. 
2 .  Grade  Structure 

The  mean  grade  level  prior  to  the  implementation  of 
APADE  was  GS  5.66.   The  mean  grade  level  measured  after 
implementation  was  GS  6.06.   This  represents  an  increase  in 
the  grade  level  in  the  Small  Purchasing  Branch  (SPB)  at  NSC 
Duarte.   With  all  other  factors  remaining  constant,  an 
increase  in  the  mean  grade  level  would  reflect  an  increase 
in  inputs  while  output  remained  constant.   Productivity 
would  decrease  in  this  scenario  as  measured  by  the  ratio  of 
outputs  to  inputs . 

There  was  an  increase  in  the  proportion  of  buyers  to 
administrative  support  personnel.   Buyers  made  up  69.9 
percent  of  the  pre  APADE  work  force  while  comprising  79.4 
percent  after  implementation.   The  mean  grade  level  for 
buyers  before  APADE  was  GS  6.04  and  after  was  GS  6.36.   The 
mean  grade  level  for  administrative  support  personnel  before 
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APADE  was  GS  4.10  and  after  was  GS  4.55.   Since  there  was 
not  a  statistically  significant  increase  in  grade  level  for 
buyers  or  administrative  support  personnel  between  the  two 
periods,  the  overall  increase  in  GS  level  in  the  post  APADE 
period  is  due  to  the  increase  in  the  proportion  of  buyers 
caused  by  the  decrease  in  the  number  of  administrative 
support  personnel  in  the  work  force. 

As  workload  decreases,  one  would  expect  to  see  an 
eventual  decrease  in  the  overall  workforce.   Although 
management  would  not  necessarily  terminate  buyers  and 
administrative  support  personnel  at  the  same  time,  the 
number  of  personnel  in  both  categories  would  be  expected  to 
decrease  to  levels  in  proportion  to  those  at  the  pre-APADE 
levels.   That  was  not  the  case  in  this  study.   APADE  shifted 
some  of  the  administrative  support  personnel  tasks  to  the 
buyers.  (See  Appendix  A) 

While  the  inclination  would  be  to  believe  that  the 
accompanying  increase  in  grade  structure  would  indicate  a 
drop  in  productivity,  further  inspection  reveals  that  this 
is  not  necessarily  the  case.   The  increase  in  grade 
structure  is  primarily  due  to  the  decrease  in  personnel, 
with  a  proportionally  larger  decrease  in  administrative 
support  personnel  than  in  buyers.   Therefore,  all  that  can 
be  said  about  the  grade  structure  is  that  APADE  shifted  some 
of  the  tasks  done  by  administrative  support  personnel  in  the 
pre-APADE  period  to  the  buyers  in  the  post  implementation 
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period.   This  shift  is  an  expected  outcome  with  the 
application  of  technology  into  the  work  environment. 
3 .  Overtime 

The  mean  amount  of  overtime  time  used  per  pay  period 
for  all  personnel  prior  to  the  implementation  of  APADE  was 
519.5  hours.   The  mean  amount  of  overtime  used  per  worker 
per  pay  period  was  6.7  hours.   The  mean  amount  of  overtime 
used  per  pay  period  for  all  personnel  after  implementation 
was  124.4  hours.   The  mean  amount  used  per  worker  per  pay 
period  was  2.0  hours.   This  represents  a  decrease  of  7  6.1 
percent  in  the  number  of  hours  of  overtime  used  per  period, 
and  a  69.8  percent  decrease  in  the  number  of  hours  used  per 
person  per  pay  period.   All  other  factors  remaining 
constant,  a  decrease  in  overtime  would  reflect  a  decrease  in 
inputs,  reflecting  an  increase  in  productivity. 

The  post  APADE  management  policy  to  use  compensatory 
time  in  lieu  of  overtime  is  a  management  decision  rather 
than  a  product  of  APADE.   However,  even  if  compensatory  time 
earned  were  added  to  overtime,  there  would  still  be  a 
decrease  of  57.5  percent  per  person  per  pay  period. 

B .   OUTPUTS 

1 .  Number  of  Purchase  Orders 

The  mean  number  of  purchase  orders  processed  during 
the  year  prior  to  the  implementation  of  APADE  was  7123  per 
month.   The  mean  number  of  purchase  orders  processed  during 
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the  year  after  the  implementation  was  4  992  per  month.   This 
represents  a  decrease  of  29.9  percent  in  the  number  of 
purchase  orders  processed  per  month.   This  is  primarily  a 
reflection  of  a  policy  which  directed  previous  NSC  Duarte 
customers  to  other  sources  of  supply. 

2 .  Procurement  Action  Lead  Time  (PALT) 

The  mean  Procurement  Action  Lead  Time  (PALT)  during 
the  year  prior  to  the  implementation  of  APADE  was  52.4  days. 
The  mean  PALT  during  the  year  after  the  implementation  was 
23.5  days.   This  represents  a  decrease  of  55.2  percent  in 
the  PALT  value  between  the  two  years.   All  other  factors 
remaining  constant  a  decrease  in  PALT  would  indicate  that 
some  degree  of  productivity  improvement  has  been  attained. 

There  was  a  29.9  percent  decrease  in  purchase 
requests  at  NSC  Duarte  between  the  two  periods  measured. 
Also,  the  ratio  of  transactions  processed  to  the  number  of 
labor  hours  worked  remained  statistically  constant  between 
the  two  periods.   This  would  tend  to  indicate  that  the 
decrease  in  PALT  was  primarily  due  to  the  decrease  in 
workload. 

The  document  flow  charts  provided  by  NSC  Duarte  for 
the  before  and  after  APADE  periods  (Appendix  A)  reflect  a 
significant  change  in  the  document  flow  process.   The  flow 
of  documentation  before  APADE  required  54  document  movements 
between  various  clerks,  supervisors,  and  procurement 
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personnel.   After  the  implementation  of  APADE  the  new 
document  flow  process  required  only  15  document  movements. 
Although  APADE  may  not  have  improved  the  productivity  of  the 
personnel  individually,  it  did  streamline  the  procurement 
process  and  reduced  the  number  of  potential  delays  in  the 
system. 4 

Interviews  with  Small  Purchasing  Branch  (SPB) 
personnel  revealed  that  they  believed  that  the 
implementation  of  APADE  was  the  significant  factor  in  the 
reduction  of  PALT.   Although  there  was  a  29.9  percent 
decrease  in  the  number  of  purchase  requests  between  the 
before  and  after  periods,  there  was  also  a  15.0  percent 
decrease  in  staff  size. 

We  can  not  say  with  any  quantitative  certainty  the 
extent  to  which  APADE  affected  PALT.   However,  it  can  be 
said  that  APADE  played  the  primary  role  in  the  reduction  of 
the  document  flow  within  the  procurement  process.    Although 
it  is  our  opinion  that  PALT  was  dramatically  reduced  at  NSC 
Duarte  due  to  the  implementation  of  APADE,  no  conclusive 
determination  can  be  made  to  ascertain  the  degree  of 
influence  either  the  implementation  of  APADE  or  the  decrease 
in  workload  has  had  on  PALT. 


Potential  delays  would  be  defined  as  time  waiting  in  inboxes, 
outboxes,  going  from  desk  to  desk,  awaiting  reviews,  signatures, 
etc.  . 
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C.   SOCIAL  EFFECTS 
1 .  Annual  Leave 

The  mean  amount  of  annual  leave  used  per  pay  period 
for  all  personnel  prior  to  the  implementation  of  APADE  was 
442.22  hours.   The  mean  amount  of  annual  leave  used  per 
person  per  pay  period  was  5.40  hours.   The  mean  amount  of 
annual  leave  used  per  pay  period  for  all  personnel  after 
implementation  was  380.05  hours.   The  mean  amount  of  annual 
leave  used  per  worker  per  pay  period  was  6.12  hours.   This 
represents  a  decrease  of  14.10  percent  in  the  number  of 
hours  of  annual  leave  taken  per  period  and  a  13.3  percent 
increase  in  the  number  of  hours  per  person  per  pay  period. 

The  amount  of  annual  leave  used  per  pay  period  for 
all  personnel  is  not  an  accurate  measure  of  degree  of  annual 
leave  taken  in  that  it  does  not  consider  the  fact  that  there 
were  fewer  staff  members  after  APADE  than  before. 
Therefore,  the  appropriate  measure  is  the  leave  taken  per 
person  per  pay  period  which  normalizes  the  relationship. 

All  else  being  equal,  an  increase  in  annual  leave 
taken  would  indicate  an  increase  in  management's  inclination 
or  ability  to  allow  personnel  to  take  leave.   The  increase 
in  the  amount  of  leave  taken  would  act  to  decrease  overall 
worker  input  hours,  thus  when  taken  by  itself,  could 
indirectly  imply  an  increase  in  productivity.   Because  of 
the  reduction  in  workload,  no  conclusive  determination  can 
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be  made  to  ascertain  the  degree  of  influence  either  the 
implementation  of  APADE  or  the  decrease  in  workload  has  had 
on  annual  leave. 
2 .  Sick  Leave 

Sick  leave  is  a  classic  objective  indicator  of 
morale  -  it  being  the  safety  valve  beyond  management 
scheduling.   The  mean  amount  of  sick  leave  used  pay  period 
for  all  personnel  prior  to  the  implementation  of  APADE  was 
255.83  hours.   The  mean  amount  of  sick  leave  used  per  person 
per  pay  period  was  3.15  hours.   The  mean  amount  of  sick 
leave  used  per  pay  period  for  all  personnel  after 
implementation  was  262.03  hours.   The  mean  amount  of  sick 
leave  used  per  worker  per  pay  period  was  4.20  hours.   This 
represents  an  increase  of  2.40  percent  in  the  number  of 
hours  of  sick  leave  taken  per  period  and  a  33.20  percent 
increase  in  the  number  of  hours  per  person  per  pay  period  by 
SPB  personnel . 

During  the  analysis  of  the  sick  leave  used  per 
worker  per  pay  period,  it  was  noted  that  the  sick  leave 
trend  analysis  remained  virtually  unchanged  throughout  the 
two  periods  except  during  one  pay  period  in  the  post 
implementation  period.   This  period  included  a  major  natural 
disaster.   During  this  period  that  the  amount  of  sick  leave 
taken  increased  significantly.   Although  this  disaster  did 
not  physically  affect  the  APADE  system,  it  can  be  inferred 
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that  there  was  a  noted  increase  in  the  stress  and  anxiety 
felt  by  workers  during  the  period. 

Taken  by  itself,  an  increase  in  sick  leave  would 
indicate  that  there  was  a  decrease  in  worker  satisfaction. 
However,  due  to  the  environmental  factors  beyond  the  control 
of  NSC  Duarte's  management,  which  accounted  for  a 
significant  portion  of  that  increase,  the  increase  noted 
cannot  be  attributed  to  a  decrease  in  worker  satisfaction. 
3 .  Leave  Without  Pay 

The  mean  amount  of  leave  without  pay  used  per  pay 
period  for  all  personnel  prior  to  the  implementation  of 
APADE  was  155.4  hours.   The  mean  amount  of  leave  without  pay 
used  per  person  per  pay  period  was  1.87  hours.   The  mean 
amount  of  leave  without  pay  used  per  pay  period  for  all 
personnel  after  implementation  was  112.05  hours.   The  mean 
amount  of  leave  without  pay  used  per  worker  per  pay  period 
was  1.80  hours.   This  represents  a  decrease  of  27.90  percent 
in  the  number  of  hours  of  leave  without  pay  taken  per  period 
and  a  3.80  percent  decrease  in  the  number  of  hours  per 
person  per  pay  period  by  SPB  personnel. 

This  is  a  combined  worker  and  management -driven 
social  factor.   A  worker  can  request  LWOP,  but  authorization 
is  up  to  the  discretion  of  management.   All  else  being 
equal,  a  decrease  in  the  amount  of  LWOP  taken  per  worker 
could  indicate  a  'not  unhappy'  condition  with  the  work 
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place,  thus  indicating  an  increase  in  worker  satisfaction  in 
the  work  environment.   This  would  have  a  positive  overall 
effect  on  productivity.   In  this  study,  there  was  no 
statistically  significant  increase/decrease  in  LWOP  and 
therefore  the  only  conclusion  that  can  be  drawn  is  that  the 
implementation  of  APADE  did  not  affect  LWOP  in  the  work 
environment . 

D.  PRODUCTIVITY  MEASUREMENT 

The  mean  number  of  purchase  orders  processed  per  labor 
hour  for  the  year  prior  to  the  implementation  of  APADE  was 
0.5564.   The  mean  number  of  purchase  orders  processed  per 
labor  hour  for  the  year  after  the  implementation  was  0.5236. 
This  represents  a  5.9  percent  drop  in  purchase  orders 
processed  per  labor  hour.   Although  there  is  a  decrease  in 
productivity,  as  measured  by  purchase  orders  processed  per 
labor  hour,  the  difference  is  not  statistically  significant. 
Therefore,  the  result  is  a  productivity  measurement  that  has 
remained  constant  between  the  two  periods . 

E.  ACCOMPLISHMENTS  OF  THE  STUDY 

We  applied  a  before  and  after  pre-experimental  design 
(Campbell  and  Stanley,  1966)  that  accumulated  historical 
indicators  of  inputs,  outputs,  and  social  effects  of  office 
automation  in  an  organization.   The  study  of  the  effects  of 
the  installation  of  APADE  has  further  established  a 
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quantitative  bench  mark  for  office  automation  productivity. 
This  was  accomplished  by  using  a  standard  input/output  model 
of  productivity. 

In  this  study,  we  also  captured  empirical  data  on 
the  social  impact  of  APADE  from  objective  historical 
documentation  rather  than  through  a  subjective  user 
satisfaction  survey. 

F.   SUMMARY  OF  APADE  BENEFITS 

The  APADE  system  implemented  at  the  Small  Purchasing 
Branch  (SPB)  at  NSC  Duarte  was  found  to  enhance  the 
efficiency  of  the  procurement  process  in  a  variety  of  ways: 


1.  Dramatically  decreased  the  amount  of  document  movement 
in  the  procurement  mechanism,  thus  eliminating  possible 
delay  points  in  the  process. 

2.  Allows  for  automated  documentation  preparation,  thus 
reducing  the  requirement  for  significant  administrative 
( clerical )  support . 

3 .  Provides  for  automated  document  control  and  tracking 
status  to  procurement  personnel  and  allows  customers 
access  to  the  system  to  obtain  status  on  their  purchase 
requests.   This  frees  up  procurement  personnel  from 
having  to  answer  phone  calls  from  customers  inquiring 
on  purchase  request  status. 

4 .  Provides  real  time  contracting  information  to 
management  which  allows  them  to  more  effectively  manage 
buyer  resources  and  distribute  procurement  work  load. 

5.  The  process  regulates  the  procurement  personnel  and 
work  environment.   Management  can  assign  buys  to 
specific  buyers  electronically  and  all  purchase  orders 
can  be  easily  traced  by  management  and  buyer  personnel. 
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G.   DIRECTION  FOR  FURTHER  RESEARCH 

The  following  is  a  listing  of  potential  topics  for 
further  research: 


1.  What  has  been  the  effect  of  APADE  on  all  of  the  major 
supply  stock  points  where  it  has  been  installed?  Are 
the  results  similar  to  those  found  in  this  study? 

2.  APADE' s  primary  design  intent  was  to  improve  the 
buyer' s  ability  to  make  quality  buys  in  accordance  with 
established  rules,  regulations,  and  guidelines.   Has 
APADE  satisfied  this  intention?   Can  this  type  of 
information  be  quantitatively  measured? 

3.  What  has  been  the  effect  of  APADE  on  the  users?   Do  the 
users'  view  APADE  as  a  positive  influence  on  the 
procurement  process?   How  has  user  perception/attitude 
affected  implementation  of  APADE  at  various  sites.   How 
can  this  type  of  information  be  quantitatively 
measured? 

4.  APADE  is  currently  being  implemented  in  the  large 
procurement  process  system  (purchases  >  $25,000).   A 
study  is  needed  to  measure  the  before  and  after  effects 
of  APADE  on  the  large  procurement  cycle. 

5.  A  comparative  study  is  needed  on  the  various  automated 
procurement  systems  currently  being  utilized  by  the 
federal  government.   Under  what  conditions  would  one 
system  be  the  preferred  choice  over  the  others?   Is  it 
appropriate  to  compare  systems  with  different  designs 
on  a  single  set  of  measurement  criteria? 

6.  A  before  and  after  productivity  study  on  the  desktop 
version  of  APADE  currently  under  development  for  use  at 
small  procurement  sites  would  be  useful. 

7.  A  follow-up  to  this  study  on  the  effects  of  APADE  at 
the  NSC  Duarte  site  is  appropriate  to  determine  if 
further  improvements  in  the  productivity  are  recognized 
as  SPB  personnel  become  more  familiar  with  the  system. 

8.  A  study  to  perform  a  regression  analysis  on  those 
factors  which  may  have  had  an  affect  on  the  various 
inputs,  outputs,  and  social  effects  addressed  in  this 
study . 
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In  today's  environment  of  diminishing  resources  it  has 
become  critical  to  quantitatively  measure  the  effects  of 
decisions  on  the  organization.   This  has  never  been  more 
important  than  it  is  today  with  automation  of  the  white 
collar  work  place. 

In  determining  whether  an  organization  has  obtained  an 
advantage  or  "got  its  money' s  worth"  out  of  procuring  an 
automated  system,   planning  is  a  critical  factor.   If  an 
organization  has  made  the  decision  to  automate  or  replace  an 
existing  system,  the  organization  must  first  decide  what 
they  want  to  gain  from  the  expenditure. 

Once  the  reason  for  the  procurement  has  been  determined, 
quantitative  measures  must  be  established  to  compare  the 
before  and  after  effects  of  the  decision.   The  establishment 
of  these  measures  and  data  collection  must  start  early  in 
the  new  system's  development  phase.   Management  is 
challenged  with  using  not  only  common  'number  crunching' 
type  assessments,  but  also  measures  that  can  be  used  to 
evaluate  the  evolving  nature  of  the  organization.   One  would 
hope  that  quantitative  and  objective  qualitative  measures 
exist,  or  can  be  developed,  to  fulfill  this  requirement. 

During  implementation  of  the  new  system,  the 
measurements  should  continue,  if  nothing  more  than  to 
observe  the  trend  over  the  implementation  phase  and  provide 
a  better  understanding  of  the  dynamics  of  the  implementation 
process.   These  measures  should  continue  well  into  the  post 
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implementation  phase,  to  measure  and  note  the  trends  in  the 
growth  or  decline  of  the  system. 

Unfortunately,  the  decision  to  evaluate  the 
effectiveness  of  a  system  often  occur  after  the  system  has 
already  been  implemented.   At  this  point,  it  is  often  too 
late  to  get  an  accurate  picture.   Another  mistake  occurs 
when  a  system  is  evaluated  against  criteria  that  it  was  not 
designed  to  comply  with.   In  the  real  world  decisions  to 
measure  the  'effectiveness'  of  a  system  are  made  after  the 
development,  implementation,  and  a  significant  portion  of 
the  maintenance  dollars  have  become  sunk  costs. 

In  reviewing  the  literature  on  this  subject,  it  has  been 
found  that  great  expectations  are  given  prior  to,  during, 
and  after  the  development  of  new  systems.   Measurements  are 
not  taken  until  after  the  question  arises  whether  or  not  the 
'new'  system  in  place  actually  fulfilled  the  needs  and 
requirements  of  the  organization. 

Such  is  the  case  with  this  study.   In  conversations  with 
individuals  who  were  connected  with  the  APADE  project,  it 
was  realized  that  the  system  was  not  developed  with  a  direct 
aim  towards  increasing  productivity.   The  primary  concern  of 
the  APADE  project  was  to  develop  an  environment  which  would 
assist  contracting  personnel  in  the  compliance  with  existing 
contract  laws  and  regulations  and  provide  an  automated 
climate  that  would  evolve  with  the  ever  changing  procurement 
environment . 
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There  may  be  those  who  read  this  study  and  attempt  to 
use  it  out  of  context  to  determine  whether  or  not  the  U.  S. 
Navy  "got  it's  money's  worth'  out  of  the  APADE  system.   This 
is  not  the  intent  of  this  study.   This  study's  intention  was 
to  further  develop  a  methodology  to  compare  automated 
procurement  systems  based  on  specific  productivity  measures. 
We  believe  that  has  been  done.   However,  additional  research 
will  be  needed  to  establish  a  precise  methodology  to  further 
advance  this  notion  of  quantitatively  measuring  the 
effectiveness  of  automated  systems. 
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APPENDIX  A:  DOCUMENT  FLOW 

DOCUMENT  FLOW 
The  diagrams  that  follow  have  been  reproduced  from 
original  documents  provided  by  the  Small  Purchase  Branch  at 
the  Navy  Field  Contracting  Activity  where  this  study  was 
conducted.   Pages  75  through  81  represent  the  document  flow 
prior  to  implementing  APADE.   Pages  82  and  83  represent  the 
document  flow  using  APADE. 
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APPENDIX  B:  SUMMARY  OF  INPUT  BY  PAY  PERIOD 
PRE-APADE  IMPLEMENTATION 


NUMBER  OF 

AVERAGE 

PERIOD 

EMPLOYEES 

GRADE  LEVEL 

OVERTIME 

1 

68 

5.794 

0 

2 

68 

5.794 

0 

3 

68 

5.794 

0 

4 

68 

5.794 

0 

5 

68 

5.794 

471 

6 

70 

5.757 

368 

7 

71 

5.718 

733 

8 

72 

5.681 

179 

9 

72 

5.681 

262 

10 

73 

5.685 

565 

11 

73 

5.685 

621 

12 

73 

5.685 

502 

13 

73 

5.685 

410 

14 

74 

5.649 

720 

15 

74 

5.649 

812 

16 

74 

5.649 

852 

17 

75 

5.613 

500 

18 

75 

5.613 

926 

19 

75 

5.613 

589 

20 

76 

5.579 

1065 

21 

77 

5.546 

573 

22 

78 

5.513 

405 

23 

78 

5.513 

828 

24 

78 

5.513 

872 

25 

78 

5.513 

524 

26 

78 

5.513 

732 
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APPENDIX  B:   SUMMARY  OF  INPUT  BY  PAY  PERIOD 
AFTER  APADE  IMPLEMENTATION 


NUMBER  OF 

PERIOD 

EMPLOYEES 

1 

59 

2 

59 

3 

60 

4 

61 

5 

62 

6 

62 

7 

63 

8 

63 

9 

63 

10 

63 

11 

63 

12 

63 

13 

63 

14 

63 

15 

63 

16 

63 

17 

63 

18 

63 

19 

63 

20 

63 

21 

63 

22 

63 

23 

63 

24 

63 

25 

63 

26 

63 

AVERAGE 

GRADE  LEVEL 

OVERTIME 

6.068 

142 

6.068 

257 

6.050 

310 

6.049 

523 

6.048 

292 

6.048 

341 

6.064 

336 

6.064 

165 

6.064 

0 

6.064 

0 

6.064 

0 

6.064 

0 

6.064 

4 

6.064 

0 

6.064 

0 

6.064 

0 

6.064 

0 

6.064 

0 

6.064 

124 

6.064 

506 

6.064 

218 

6.064 

10 

6.064 

0 

6.064 

0 

6.064 

8 

6.064 

0 
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APPENDIX  C:  SUMMARY  OF  SOCIAL  EFFECTS  BY  PAY  PERIOD 


PRE-APADE  IMPLEMENTATION 


LEAVE 

WITHOUT 

PAY  PERIOD 

ANNUAL  LEAVE 

SICK  LEAVE 

PAY 

1 

257 

383 

44 

2 

245 

286 

57 

3 

255 

355 

52 

4 

320 

150 

116 

5 

282 

258 

73 

6 

364 

277 

77 

7 

388 

350 

124 

8 

551 

241 

116 

9 

315 

169 

132 

10 

482 

242 

161 

11 

489 

174 

236 

12 

386 

274 

157 

13 

382 

241 

150 

14 

899 

188 

108 

15 

451 

215 

117 

16 

568 

259 

165 

17 

376 

256 

167 

18 

363 

192 

210 

19 

274 

194 

112 

20 

367 

318 

156 

21 

494 

252 

242 

22 

339 

320 

261 

23 

519 

423 

247 

24 

646 

223 

377 

25 

349 

232 

239 

26 

1142 

177 

134 
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APPENDIX  C:   SUMMARY  OF  SOCIAL  EFFECTS  BY  PAY  PERIOD 


AFTER  APADE  IMPLEMENTATION 


LEAVE 

WITHOUT 

PAY  PERIOD 

ANNUAL  LEAVE 

SICK  LEAVE 

PAY 

1 

341 

302 

60 

2 

210 

275 

82 

3 

301 

366 

16 

4 

302 

351 

65 

5 

245 

418 

86 

6 

300 

324 

16 

7 

320 

357 

110 

8 

397 

155 

61 

9 

232 

210 

105 

10 

302 

280 

88 

11 

423 

204 

67 

12 

385 

167 

63 

13 

314 

202 

46 

14 

545 

261 

91 

15 

530 

291 

132 

16 

508 

261 

106 

17 

374 

301 

95 

18 

285 

219 

58 

19 

245 

243 

135 

20 

347 

229 

221 

21 

568 

311 

209 

22 

270 

325 

135 

23 

305 

225 

177 

24 

488 

313 

212 

25 

405 

134 

222 

26 

942 

81 

245 
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APPENDIX  D:  PRODUCTIVITY  MEASUREMENT  PER  PAY  PERIOD 
PRE -AP APE  IMPLEMENTATION 


TRANSACTIONS 

PER  LABOR 

PERIOD 

#  TRANSACTIONS 

LABOR  HOURS 

HOUR 

1 

2537 

5156 

0.4  92 

2 

3080 

5252 

0.587 

3 

2738 

5178 

0.529 

4 

2738 

5254 

0.521 

5 

3540 

5698 

0.621 

6 

3540 

5570 

0.636 

7 

3138 

5950 

0.527 

8 

3028 

5431 

0.558 

9 

3248 

5806 

0.559 

10 

3370 

5840 

0.577 

11 

3602 

5881 

0.613 

12 

3913 

5845 

0.669 

13 

3891 

5796 

0.671 

14 

3838 

5844 

0.657 

15 

3831 

6348 

0.604 

16 

3739 

6180 

0.605 

17 

3739 

6101 

0.613 

18 

4684 

6561 

0.714 

19 

4842 

6408 

0.756 

20 

3129 

6623 

0.472 

21 

2444 

5985 

0.408 

22 

2560 

6125 

0.418 

23 

2677 

6279 

0.426 

24 

2675 

6266 

0.427 

25 

2669 

6344 

0.421 

26 

2288 

5918 

0.387 
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APPENDIX  D:   PRODUCTIVITY  MEASUREMENT  PER  PAY  PERIOD 
AFTER  APADE  IMPLEMENTATION 


TRANSACTIONS 

PER  LABOR 

PERIOD 

#  TRANSACTIONS 

LABOR  HOURS 

HOUR 

1 

2619 

4238 

0.619 

2 

2619 

4489 

0.583 

3 

2576 

4506 

0.572 

4 

2656 

4764 

0.557 

5 

2958 

4582 

0.645 

6 

2948 

4740 

0.622 

7 

2701 

4588 

0.589 

8 

2515 

4591 

0.548 

9 

2487 

4492 

0.554 

10 

2449 

4369 

0.560 

11 

2441 

4345 

0.562 

12 

2411 

4424 

0.545 

13 

2379 

4481 

0.531 

14 

1960 

4142 

0.473 

15 

1959 

4086 

0.479 

16 

2268 

4164 

0.544 

17 

2319 

4269 

0.543 

18 

2706 

4477 

0.604 

19 

2922 

4540 

0.644 

20 

2117 

4748 

0.446 

21 

1312 

4169 

0.315 

22 

1476 

4319 

0.342 

23 

1884 

4332 

0.435 

24 

1848 

4026 

0.459 

25 

1632 

4286 

0.381 

26 

1748 

3771 

0.464 
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